2023-06-28T19:47:07.278 INFO:root:teuthology version: 0.0.1.dev132+gaa9345a 2023-06-28T19:47:07.279 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_vshankar@teuthology --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} --archive /home/teuthworker/archive/vshankar-2023-06-28_10:04:29-fs-wip-vshankar-testing-20230628.043830-testing-default-smithi/7319991 --name vshankar-2023-06-28_10:04:29-fs-wip-vshankar-testing-20230628.043830-testing-default-smithi -- /home/teuthworker/archive/vshankar-2023-06-28_10:04:29-fs-wip-vshankar-testing-20230628.043830-testing-default-smithi/7319991/orig.config.yaml 2023-06-28T19:47:07.323 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-28T19:47:07.429 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2023-06-28_10:04:29-fs-wip-vshankar-testing-20230628.043830-testing-default-smithi/7319991 branch: wip-vshankar-testing-20230628.043830 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} email: vshankar@redhat.com first_in_suite: false job_id: '7319991' kernel: client: sha1: distro flavor: default kdb: true sha1: d535508c699c563d11e77fb7b284c600a069451a ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2023-06-28_10:04:29-fs-wip-vshankar-testing-20230628.043830-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-vshankar-testing-20230628.043830 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: a7bcf4acb88d3aafb511a5f395764c81b8c88a7c ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a7bcf4acb88d3aafb511a5f395764c81b8c88a7c kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20230628.043830 sha1: a7bcf4acb88d3aafb511a5f395764c81b8c88a7c owner: scheduled_vshankar@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: a7bcf4acb88d3aafb511a5f395764c81b8c88a7c sleep_before_teardown: 0 suite: fs suite_branch: wip-vshankar-testing-20230628.043830 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a7bcf4acb88d3aafb511a5f395764c81b8c88a7c/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a7bcf4acb88d3aafb511a5f395764c81b8c88a7c syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi178.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_quota teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: aa9345a996190b15ab9d46b5969b5f8280bfba99 timestamp: 2023-06-28_10:04:29 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-06-28T19:47:07.430 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a7bcf4acb88d3aafb511a5f395764c81b8c88a7c/qa; will attempt to use it 2023-06-28T19:47:07.431 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a7bcf4acb88d3aafb511a5f395764c81b8c88a7c/qa/tasks 2023-06-28T19:47:07.431 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-06-28T19:47:07.434 INFO:teuthology.task.internal:Checking packages... 2023-06-28T19:47:07.471 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'a7bcf4acb88d3aafb511a5f395764c81b8c88a7c' 2023-06-28T19:47:07.471 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-06-28T19:47:07.472 INFO:teuthology.packaging:ref: None 2023-06-28T19:47:07.472 INFO:teuthology.packaging:tag: None 2023-06-28T19:47:07.472 INFO:teuthology.packaging:branch: wip-vshankar-testing-20230628.043830 2023-06-28T19:47:07.473 INFO:teuthology.packaging:sha1: a7bcf4acb88d3aafb511a5f395764c81b8c88a7c 2023-06-28T19:47:07.473 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-vshankar-testing-20230628.043830 2023-06-28T19:47:17.616 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4628.ga7bcf4ac 2023-06-28T19:47:17.616 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-06-28T19:47:17.623 INFO:teuthology.task.internal:no buildpackages task found 2023-06-28T19:47:17.624 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-06-28T19:47:17.637 INFO:teuthology.task.internal:Saving configuration 2023-06-28T19:47:17.653 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-06-28T19:47:17.664 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-28T19:47:27.698 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-06-28_10:04:29-fs-wip-vshankar-testing-20230628.043830-testing-default-smithi/7319991', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-28 19:41:37.828967', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-06-28T19:47:32.727 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-06-28_10:04:29-fs-wip-vshankar-testing-20230628.043830-testing-default-smithi/7319991', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-28 19:41:37.830701', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-06-28T19:47:32.727 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-06-28T19:47:32.734 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-06-28T19:47:32.734 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-06-28T19:47:32.734 INFO:teuthology.run_tasks:Running task console_log... 2023-06-28T19:47:42.834 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6fa3879040>, signals=[15]) 2023-06-28T19:47:42.835 INFO:teuthology.run_tasks:Running task internal.connect... 2023-06-28T19:47:42.847 INFO:teuthology.task.internal:Opening connections... 2023-06-28T19:47:42.847 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi043.front.sepia.ceph.com 2023-06-28T19:47:42.849 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T19:47:47.919 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi178.front.sepia.ceph.com 2023-06-28T19:47:47.920 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T19:47:47.990 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-06-28T19:47:48.006 DEBUG:teuthology.orchestra.run.smithi043:> uname -m 2023-06-28T19:47:48.025 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2023-06-28T19:47:48.025 DEBUG:teuthology.orchestra.run.smithi043:> cat /etc/os-release 2023-06-28T19:47:48.082 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="Red Hat Enterprise Linux" 2023-06-28T19:47:48.083 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="8.6 (Ootpa)" 2023-06-28T19:47:48.083 INFO:teuthology.orchestra.run.smithi043.stdout:ID="rhel" 2023-06-28T19:47:48.083 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="fedora" 2023-06-28T19:47:48.083 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="8.6" 2023-06-28T19:47:48.083 INFO:teuthology.orchestra.run.smithi043.stdout:PLATFORM_ID="platform:el8" 2023-06-28T19:47:48.083 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-06-28T19:47:48.083 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31" 2023-06-28T19:47:48.084 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-06-28T19:47:48.084 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://www.redhat.com/" 2023-06-28T19:47:48.084 INFO:teuthology.orchestra.run.smithi043.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-28T19:47:48.084 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-28T19:47:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T19:47:48.084 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-28T19:47:48.084 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-06-28T19:47:48.085 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-28T19:47:48.085 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-06-28T19:47:48.085 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2023-06-28T19:47:53.119 DEBUG:teuthology.orchestra.run.smithi178:> uname -m 2023-06-28T19:47:53.155 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2023-06-28T19:47:53.155 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2023-06-28T19:47:53.215 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="Red Hat Enterprise Linux" 2023-06-28T19:47:53.215 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="8.6 (Ootpa)" 2023-06-28T19:47:53.215 INFO:teuthology.orchestra.run.smithi178.stdout:ID="rhel" 2023-06-28T19:47:53.215 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="fedora" 2023-06-28T19:47:53.215 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="8.6" 2023-06-28T19:47:53.215 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el8" 2023-06-28T19:47:53.216 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-06-28T19:47:53.216 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31" 2023-06-28T19:47:53.216 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-06-28T19:47:53.216 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://www.redhat.com/" 2023-06-28T19:47:53.216 INFO:teuthology.orchestra.run.smithi178.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-28T19:47:53.216 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-28T19:47:53.217 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T19:47:53.217 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-28T19:47:53.217 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-06-28T19:47:53.217 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-28T19:47:53.217 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-06-28T19:47:53.217 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2023-06-28T19:47:57.027 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-06-28T19:47:57.045 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-06-28T19:47:57.056 INFO:teuthology.task.internal:Checking for old test directory... 2023-06-28T19:47:57.056 DEBUG:teuthology.orchestra.run.smithi043:> test '!' -e /home/ubuntu/cephtest 2023-06-28T19:47:57.059 DEBUG:teuthology.orchestra.run.smithi178:> test '!' -e /home/ubuntu/cephtest 2023-06-28T19:47:57.078 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-06-28T19:47:57.089 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-06-28T19:47:57.089 DEBUG:teuthology.orchestra.run.smithi043:> test -z $(ls -A /var/lib/ceph) 2023-06-28T19:47:57.119 DEBUG:teuthology.orchestra.run.smithi178:> test -z $(ls -A /var/lib/ceph) 2023-06-28T19:47:57.149 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-06-28T19:47:57.210 INFO:teuthology.run_tasks:Running task kernel... 2023-06-28T19:47:57.228 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'd535508c699c563d11e77fb7b284c600a069451a'} 2023-06-28T19:47:57.229 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-06-28T19:47:57.229 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-06-28T19:47:57.230 DEBUG:teuthology.orchestra.run.smithi043:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-28T19:47:57.231 DEBUG:teuthology.orchestra.run.smithi178:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-28T19:47:57.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-28T19:47:57.250 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2023-06-28T19:47:57.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-28T19:47:57.251 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2023-06-28T19:47:57.309 INFO:teuthology.orchestra.run.smithi178.stdout:4.18.0-372.9.1.el8.x86_64 2023-06-28T19:47:57.309 INFO:teuthology.task.kernel:Running kernel on smithi178: 4.18.0-372.9.1.el8.x86_64 2023-06-28T19:47:57.310 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum install -y kernel 2023-06-28T19:47:57.312 INFO:teuthology.orchestra.run.smithi043.stdout:4.18.0-372.9.1.el8.x86_64 2023-06-28T19:47:57.312 INFO:teuthology.task.kernel:Running kernel on smithi043: 4.18.0-372.9.1.el8.x86_64 2023-06-28T19:47:57.312 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum install -y kernel 2023-06-28T19:47:58.526 INFO:teuthology.orchestra.run.smithi043.stdout:Updating Subscription Management repositories. 2023-06-28T19:47:58.526 INFO:teuthology.orchestra.run.smithi043.stdout:Unable to read consumer identity 2023-06-28T19:47:58.529 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-06-28T19:47:58.530 INFO:teuthology.orchestra.run.smithi178.stdout:Unable to read consumer identity 2023-06-28T19:48:02.073 INFO:teuthology.orchestra.run.smithi043.stdout:Copr repo for python3-asyncssh owned by ceph 1.5 kB/s | 3.5 kB 00:02 2023-06-28T19:48:02.323 INFO:teuthology.orchestra.run.smithi178.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-28T19:48:04.598 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 6.2 MB/s | 16 MB 00:02 2023-06-28T19:48:04.873 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 6.2 MB/s | 16 MB 00:02 2023-06-28T19:48:08.173 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 369 kB/s | 24 kB 00:00 2023-06-28T19:48:08.298 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 330 kB/s | 24 kB 00:00 2023-06-28T19:48:08.884 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-06-28T19:48:08.941 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2023-06-28T19:48:08.942 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2023-06-28T19:48:08.942 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2023-06-28T19:48:09.026 INFO:teuthology.orchestra.run.smithi178.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-06-28T19:48:09.064 DEBUG:teuthology.orchestra.run.smithi043:> echo no | sudo yum reinstall kernel || true 2023-06-28T19:48:09.081 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-06-28T19:48:09.082 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2023-06-28T19:48:09.082 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-06-28T19:48:09.196 DEBUG:teuthology.orchestra.run.smithi178:> echo no | sudo yum reinstall kernel || true 2023-06-28T19:48:09.413 INFO:teuthology.orchestra.run.smithi043.stdout:Updating Subscription Management repositories. 2023-06-28T19:48:09.414 INFO:teuthology.orchestra.run.smithi043.stdout:Unable to read consumer identity 2023-06-28T19:48:09.526 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-06-28T19:48:09.526 INFO:teuthology.orchestra.run.smithi178.stdout:Unable to read consumer identity 2023-06-28T19:48:09.683 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:01 ago on Wed 28 Jun 2023 07:48:08 PM UTC. 2023-06-28T19:48:09.729 INFO:teuthology.orchestra.run.smithi043.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-06-28T19:48:09.765 INFO:teuthology.orchestra.run.smithi043.stderr:Error: No packages marked for reinstall. 2023-06-28T19:48:09.781 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:01 ago on Wed 28 Jun 2023 07:48:08 PM UTC. 2023-06-28T19:48:09.812 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum reinstall -y kernel || true 2023-06-28T19:48:09.824 INFO:teuthology.orchestra.run.smithi178.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-06-28T19:48:09.866 INFO:teuthology.orchestra.run.smithi178.stderr:Error: No packages marked for reinstall. 2023-06-28T19:48:09.912 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum reinstall -y kernel || true 2023-06-28T19:48:10.162 INFO:teuthology.orchestra.run.smithi043.stdout:Updating Subscription Management repositories. 2023-06-28T19:48:10.163 INFO:teuthology.orchestra.run.smithi043.stdout:Unable to read consumer identity 2023-06-28T19:48:10.242 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-06-28T19:48:10.242 INFO:teuthology.orchestra.run.smithi178.stdout:Unable to read consumer identity 2023-06-28T19:48:10.426 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:02 ago on Wed 28 Jun 2023 07:48:08 PM UTC. 2023-06-28T19:48:10.468 INFO:teuthology.orchestra.run.smithi043.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-06-28T19:48:10.492 INFO:teuthology.orchestra.run.smithi043.stderr:Error: No packages marked for reinstall. 2023-06-28T19:48:10.494 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:02 ago on Wed 28 Jun 2023 07:48:08 PM UTC. 2023-06-28T19:48:10.534 INFO:teuthology.orchestra.run.smithi178.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-06-28T19:48:10.538 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q kernel | sort -rV | head -n 1 2023-06-28T19:48:10.581 INFO:teuthology.orchestra.run.smithi178.stderr:Error: No packages marked for reinstall. 2023-06-28T19:48:10.620 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-06-28T19:48:10.621 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-06-28T19:48:10.621 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-06-28T19:48:10.621 DEBUG:teuthology.parallel:result is None 2023-06-28T19:48:10.632 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q kernel | sort -rV | head -n 1 2023-06-28T19:48:10.712 INFO:teuthology.orchestra.run.smithi178.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-06-28T19:48:10.713 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-06-28T19:48:10.713 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-06-28T19:48:10.713 DEBUG:teuthology.parallel:result is None 2023-06-28T19:48:10.713 INFO:teuthology.run_tasks:Running task internal.base... 2023-06-28T19:48:10.723 INFO:teuthology.task.internal:Creating test directory... 2023-06-28T19:48:10.724 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-28T19:48:10.726 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-28T19:48:10.743 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-06-28T19:48:10.752 INFO:teuthology.run_tasks:Running task internal.archive... 2023-06-28T19:48:10.765 INFO:teuthology.task.internal:Creating archive directory... 2023-06-28T19:48:10.766 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-28T19:48:10.782 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-28T19:48:10.809 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-06-28T19:48:10.825 INFO:teuthology.task.internal:Enabling coredump saving... 2023-06-28T19:48:10.825 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-28T19:48:10.841 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-28T19:48:10.877 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-28T19:48:10.878 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-28T19:48:10.890 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-28T19:48:10.894 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-28T19:48:10.896 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-06-28T19:48:10.942 INFO:teuthology.task.internal:Configuring sudo... 2023-06-28T19:48:10.943 DEBUG:teuthology.orchestra.run.smithi043:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-28T19:48:10.945 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-28T19:48:10.971 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-06-28T19:48:10.981 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-06-28T19:48:10.981 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-28T19:48:11.013 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-28T19:48:11.029 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-28T19:48:15.532 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-28T19:48:15.571 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-28T19:48:15.658 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-28T19:48:15.703 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T19:48:15.704 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-28T19:48:15.774 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-28T19:48:15.825 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-28T19:48:15.877 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-28T19:48:15.965 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-28T19:48:16.008 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T19:48:16.008 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-28T19:48:16.077 DEBUG:teuthology.orchestra.run.smithi043:> sudo service rsyslog restart 2023-06-28T19:48:16.079 DEBUG:teuthology.orchestra.run.smithi178:> sudo service rsyslog restart 2023-06-28T19:48:16.145 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-28T19:48:16.174 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-28T19:48:16.542 INFO:teuthology.run_tasks:Running task internal.timer... 2023-06-28T19:48:16.556 INFO:teuthology.task.internal:Starting timer... 2023-06-28T19:48:16.556 INFO:teuthology.run_tasks:Running task pcp... 2023-06-28T19:48:16.573 INFO:teuthology.run_tasks:Running task selinux... 2023-06-28T19:48:16.588 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-06-28T19:48:16.589 DEBUG:teuthology.orchestra.run.smithi043:> sudo service auditd rotate 2023-06-28T19:48:16.650 INFO:teuthology.orchestra.run.smithi043.stdout:Rotating logs: 2023-06-28T19:48:16.652 DEBUG:teuthology.orchestra.run.smithi178:> sudo service auditd rotate 2023-06-28T19:48:16.713 INFO:teuthology.orchestra.run.smithi178.stdout:Rotating logs: 2023-06-28T19:48:16.714 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-06-28T19:48:16.715 DEBUG:teuthology.orchestra.run.smithi043:> /usr/sbin/getenforce 2023-06-28T19:48:16.739 INFO:teuthology.orchestra.run.smithi043.stdout:Permissive 2023-06-28T19:48:16.740 DEBUG:teuthology.orchestra.run.smithi178:> /usr/sbin/getenforce 2023-06-28T19:48:16.764 INFO:teuthology.orchestra.run.smithi178.stdout:Permissive 2023-06-28T19:48:16.765 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi043.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi178.front.sepia.ceph.com': 'permissive'} 2023-06-28T19:48:16.765 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-28T19:48:16.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-28T19:48:16.809 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-28T19:48:16.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-28T19:48:16.836 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-06-28T19:48:16.837 DEBUG:teuthology.orchestra.run.smithi043:> sudo /usr/sbin/setenforce permissive 2023-06-28T19:48:16.879 DEBUG:teuthology.orchestra.run.smithi178:> sudo /usr/sbin/setenforce permissive 2023-06-28T19:48:16.912 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-06-28T19:48:16.922 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-06-28T19:48:16.923 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-06-28T19:48:16.944 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-06-28T19:48:16.946 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi043.front.sepia.ceph.com,smithi178.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-06-28T20:03:51.664 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi043.front.sepia.ceph.com'), Remote(name='ubuntu@smithi178.front.sepia.ceph.com')] 2023-06-28T20:03:51.666 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2023-06-28T20:03:51.667 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T20:03:51.736 DEBUG:teuthology.orchestra.run.smithi043:> true 2023-06-28T20:03:51.801 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2023-06-28T20:03:51.802 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-28T20:03:51.872 DEBUG:teuthology.orchestra.run.smithi178:> true 2023-06-28T20:03:51.943 INFO:teuthology.run_tasks:Running task clock... 2023-06-28T20:03:51.960 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-06-28T20:03:51.961 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-28T20:03:51.962 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-28T20:03:51.964 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-28T20:03:51.965 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-28T20:03:51.995 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-28T20:03:52.010 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-28T20:03:52.015 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-28T20:03:52.029 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-28T20:03:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: ntpd: command not found 2023-06-28T20:03:52.085 INFO:teuthology.orchestra.run.smithi043.stdout:506 Cannot talk to daemon 2023-06-28T20:03:52.093 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: ntpd: command not found 2023-06-28T20:03:52.099 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-28T20:03:52.105 INFO:teuthology.orchestra.run.smithi178.stdout:506 Cannot talk to daemon 2023-06-28T20:03:52.113 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-28T20:03:52.120 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-28T20:03:52.134 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-28T20:03:52.192 INFO:teuthology.orchestra.run.smithi043.stderr:bash: ntpq: command not found 2023-06-28T20:03:52.195 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-28T20:03:52.195 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2023-06-28T20:03:52.195 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-28T20:03:52.196 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-28T20:03:52.196 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-28T20:03:52.196 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-28T20:03:52.214 INFO:teuthology.orchestra.run.smithi178.stderr:bash: ntpq: command not found 2023-06-28T20:03:52.217 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-28T20:03:52.217 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2023-06-28T20:03:52.218 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-28T20:03:52.218 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-28T20:03:52.218 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-28T20:03:52.219 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-28T20:03:52.220 INFO:teuthology.run_tasks:Running task install... 2023-06-28T20:03:52.233 DEBUG:teuthology.task.install:project ceph 2023-06-28T20:03:52.234 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a7bcf4acb88d3aafb511a5f395764c81b8c88a7c'}} 2023-06-28T20:03:52.234 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a7bcf4acb88d3aafb511a5f395764c81b8c88a7c'} 2023-06-28T20:03:52.235 INFO:teuthology.task.install:Using flavor: default 2023-06-28T20:03:52.243 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-06-28T20:03:52.243 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-06-28T20:03:52.244 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a7bcf4acb88d3aafb511a5f395764c81b8c88a7c', 'tag': None, 'wait_for_package': False} 2023-06-28T20:03:52.245 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a7bcf4acb88d3aafb511a5f395764c81b8c88a7c 2023-06-28T20:03:52.247 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a7bcf4acb88d3aafb511a5f395764c81b8c88a7c', 'tag': None, 'wait_for_package': False} 2023-06-28T20:03:52.247 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a7bcf4acb88d3aafb511a5f395764c81b8c88a7c 2023-06-28T20:04:02.394 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230628.043830/a7bcf4acb88d3aafb511a5f395764c81b8c88a7c/centos/8/flavors/default/ 2023-06-28T20:04:02.394 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4628.ga7bcf4ac 2023-06-28T20:04:02.416 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230628.043830/a7bcf4acb88d3aafb511a5f395764c81b8c88a7c/centos/8/flavors/default/ 2023-06-28T20:04:02.416 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4628.ga7bcf4ac 2023-06-28T20:04:02.528 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230628.043830/a7bcf4acb88d3aafb511a5f395764c81b8c88a7c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230628.043830/a7bcf4acb88d3aafb511a5f395764c81b8c88a7c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230628.043830/a7bcf4acb88d3aafb511a5f395764c81b8c88a7c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-28T20:04:02.529 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:04:02.529 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-28T20:04:02.573 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-06-28T20:04:02.574 DEBUG:teuthology.orchestra.run.smithi178:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a7bcf4acb88d3aafb511a5f395764c81b8c88a7c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-28T20:04:02.644 DEBUG:teuthology.orchestra.run.smithi178:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-28T20:04:02.706 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230628.043830/a7bcf4acb88d3aafb511a5f395764c81b8c88a7c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230628.043830/a7bcf4acb88d3aafb511a5f395764c81b8c88a7c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230628.043830/a7bcf4acb88d3aafb511a5f395764c81b8c88a7c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-28T20:04:02.707 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:04:02.707 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-28T20:04:02.724 DEBUG:teuthology.orchestra.run.smithi178:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-28T20:04:02.742 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-06-28T20:04:02.742 DEBUG:teuthology.orchestra.run.smithi043:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a7bcf4acb88d3aafb511a5f395764c81b8c88a7c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-28T20:04:02.790 INFO:teuthology.orchestra.run.smithi178.stdout:check_obsoletes = 1 2023-06-28T20:04:02.792 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum clean all 2023-06-28T20:04:02.814 DEBUG:teuthology.orchestra.run.smithi043:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-28T20:04:02.896 DEBUG:teuthology.orchestra.run.smithi043:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-28T20:04:02.964 INFO:teuthology.orchestra.run.smithi043.stdout:check_obsoletes = 1 2023-06-28T20:04:02.965 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum clean all 2023-06-28T20:04:03.161 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-06-28T20:04:03.361 INFO:teuthology.orchestra.run.smithi043.stdout:Updating Subscription Management repositories. 2023-06-28T20:04:03.418 INFO:teuthology.orchestra.run.smithi178.stdout:44 files removed 2023-06-28T20:04:03.464 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-06-28T20:04:03.588 INFO:teuthology.orchestra.run.smithi043.stdout:44 files removed 2023-06-28T20:04:03.636 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-06-28T20:04:03.796 INFO:teuthology.orchestra.run.smithi178.stdout:Updating Subscription Management repositories. 2023-06-28T20:04:03.991 INFO:teuthology.orchestra.run.smithi043.stdout:Updating Subscription Management repositories. 2023-06-28T20:04:04.522 INFO:teuthology.orchestra.run.smithi178.stdout:ceph packages for x86_64 209 kB/s | 81 kB 00:00 2023-06-28T20:04:04.815 INFO:teuthology.orchestra.run.smithi178.stdout:ceph noarch packages 36 kB/s | 9.4 kB 00:00 2023-06-28T20:04:04.867 INFO:teuthology.orchestra.run.smithi043.stdout:ceph packages for x86_64 161 kB/s | 81 kB 00:00 2023-06-28T20:04:05.107 INFO:teuthology.orchestra.run.smithi178.stdout:ceph source packages 5.5 kB/s | 1.5 kB 00:00 2023-06-28T20:04:05.167 INFO:teuthology.orchestra.run.smithi043.stdout:ceph noarch packages 35 kB/s | 9.4 kB 00:00 2023-06-28T20:04:05.526 INFO:teuthology.orchestra.run.smithi043.stdout:ceph source packages 4.4 kB/s | 1.5 kB 00:00 2023-06-28T20:04:07.758 INFO:teuthology.orchestra.run.smithi178.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-06-28T20:04:07.885 INFO:teuthology.orchestra.run.smithi043.stdout:Copr repo for python3-asyncssh owned by ceph 1.5 kB/s | 3.5 kB 00:02 2023-06-28T20:04:08.925 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2023-06-28T20:04:09.511 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 16 MB 00:01 2023-06-28T20:04:12.375 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 428 kB/s | 24 kB 00:00 2023-06-28T20:04:13.169 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 421 kB/s | 24 kB 00:00 2023-06-28T20:04:13.293 INFO:teuthology.orchestra.run.smithi178.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 63 MB/s | 53 MB 00:00 2023-06-28T20:04:14.096 INFO:teuthology.orchestra.run.smithi043.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 62 MB/s | 53 MB 00:00 2023-06-28T20:04:21.336 INFO:teuthology.orchestra.run.smithi178.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 61 MB/s | 47 MB 00:00 2023-06-28T20:04:22.663 INFO:teuthology.orchestra.run.smithi043.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 57 MB/s | 47 MB 00:00 2023-06-28T20:04:32.350 INFO:teuthology.orchestra.run.smithi178.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-28T20:04:32.351 INFO:teuthology.orchestra.run.smithi178.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-28T20:04:32.365 INFO:teuthology.orchestra.run.smithi178.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-06-28T20:04:32.366 INFO:teuthology.orchestra.run.smithi178.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-06-28T20:04:32.367 INFO:teuthology.orchestra.run.smithi178.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-06-28T20:04:32.368 INFO:teuthology.orchestra.run.smithi178.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-28T20:04:32.370 INFO:teuthology.orchestra.run.smithi178.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-28T20:04:32.373 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-06-28T20:04:32.374 INFO:teuthology.orchestra.run.smithi178.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-06-28T20:04:32.505 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2023-06-28T20:04:32.517 INFO:teuthology.orchestra.run.smithi178.stdout:=========================================================================================================================== 2023-06-28T20:04:32.518 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2023-06-28T20:04:32.518 INFO:teuthology.orchestra.run.smithi178.stdout:=========================================================================================================================== 2023-06-28T20:04:32.518 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2023-06-28T20:04:32.518 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-06-28T20:04:32.518 INFO:teuthology.orchestra.run.smithi178.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-28T20:04:32.519 INFO:teuthology.orchestra.run.smithi178.stdout: ceph x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 6.4 k 2023-06-28T20:04:32.519 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 5.4 M 2023-06-28T20:04:32.519 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 848 k 2023-06-28T20:04:32.519 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 165 k 2023-06-28T20:04:32.519 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 1.5 M 2023-06-28T20:04:32.519 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4628.ga7bcf4ac.el8 ceph-noarch 129 k 2023-06-28T20:04:32.519 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4628.ga7bcf4ac.el8 ceph-noarch 1.7 M 2023-06-28T20:04:32.520 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4628.ga7bcf4ac.el8 ceph-noarch 7.4 M 2023-06-28T20:04:32.520 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook noarch 2:18.0.0-4628.ga7bcf4ac.el8 ceph-noarch 47 k 2023-06-28T20:04:32.520 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 13 M 2023-06-28T20:04:32.520 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 48 M 2023-06-28T20:04:32.520 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume noarch 2:18.0.0-4628.ga7bcf4ac.el8 ceph-noarch 258 k 2023-06-28T20:04:32.520 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm noarch 2:18.0.0-4628.ga7bcf4ac.el8 ceph-noarch 164 k 2023-06-28T20:04:32.520 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 258 k 2023-06-28T20:04:32.521 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top noarch 2:18.0.0-4628.ga7bcf4ac.el8 ceph-noarch 18 k 2023-06-28T20:04:32.521 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-06-28T20:04:32.521 INFO:teuthology.orchestra.run.smithi178.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-28T20:04:32.521 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-28T20:04:32.521 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 31 k 2023-06-28T20:04:32.521 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2 x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 771 k 2023-06-28T20:04:32.521 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 139 k 2023-06-28T20:04:32.521 INFO:teuthology.orchestra.run.smithi178.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-28T20:04:32.522 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-06-28T20:04:32.522 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-28T20:04:32.522 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-06-28T20:04:32.522 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-06-28T20:04:32.522 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-06-28T20:04:32.522 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 197 k 2023-06-28T20:04:32.522 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 371 k 2023-06-28T20:04:32.523 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 365 k 2023-06-28T20:04:32.523 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 106 k 2023-06-28T20:04:32.523 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 94 k 2023-06-28T20:04:32.523 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 3.4 M 2023-06-28T20:04:32.523 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 179 k 2023-06-28T20:04:32.524 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-06-28T20:04:32.524 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrading: 2023-06-28T20:04:32.524 INFO:teuthology.orchestra.run.smithi178.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-06-28T20:04:32.524 INFO:teuthology.orchestra.run.smithi178.stdout: librados2 x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 3.7 M 2023-06-28T20:04:32.524 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1 x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 3.6 M 2023-06-28T20:04:32.524 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2023-06-28T20:04:32.524 INFO:teuthology.orchestra.run.smithi178.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-28T20:04:32.525 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 22 M 2023-06-28T20:04:32.525 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4628.ga7bcf4ac.el8 ceph-noarch 23 k 2023-06-28T20:04:32.525 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 2.3 M 2023-06-28T20:04:32.525 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4628.ga7bcf4ac.el8 ceph-noarch 240 k 2023-06-28T20:04:32.525 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 4.6 M 2023-06-28T20:04:32.525 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 18 M 2023-06-28T20:04:32.525 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4628.ga7bcf4ac.el8 ceph-noarch 15 k 2023-06-28T20:04:32.526 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 24 k 2023-06-28T20:04:32.526 INFO:teuthology.orchestra.run.smithi178.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-28T20:04:32.526 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-28T20:04:32.526 INFO:teuthology.orchestra.run.smithi178.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-06-28T20:04:32.526 INFO:teuthology.orchestra.run.smithi178.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-28T20:04:32.526 INFO:teuthology.orchestra.run.smithi178.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-06-28T20:04:32.526 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 137 k 2023-06-28T20:04:32.526 INFO:teuthology.orchestra.run.smithi178.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-06-28T20:04:32.527 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-06-28T20:04:32.527 INFO:teuthology.orchestra.run.smithi178.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-28T20:04:32.527 INFO:teuthology.orchestra.run.smithi178.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-06-28T20:04:32.527 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-06-28T20:04:32.527 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-28T20:04:32.527 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1 x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 473 k 2023-06-28T20:04:32.527 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-06-28T20:04:32.528 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2 x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 6.7 M 2023-06-28T20:04:32.528 INFO:teuthology.orchestra.run.smithi178.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-28T20:04:32.528 INFO:teuthology.orchestra.run.smithi178.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-06-28T20:04:32.528 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-28T20:04:32.528 INFO:teuthology.orchestra.run.smithi178.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-28T20:04:32.528 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-28T20:04:32.528 INFO:teuthology.orchestra.run.smithi178.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-28T20:04:32.529 INFO:teuthology.orchestra.run.smithi178.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-28T20:04:32.529 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-06-28T20:04:32.529 INFO:teuthology.orchestra.run.smithi178.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-28T20:04:32.529 INFO:teuthology.orchestra.run.smithi178.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-28T20:04:32.529 INFO:teuthology.orchestra.run.smithi178.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-28T20:04:32.529 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-28T20:04:32.529 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-28T20:04:32.530 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-28T20:04:32.530 INFO:teuthology.orchestra.run.smithi178.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-28T20:04:32.530 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-28T20:04:32.530 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 45 k 2023-06-28T20:04:32.530 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 119 k 2023-06-28T20:04:32.530 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-28T20:04:32.530 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-06-28T20:04:32.531 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-28T20:04:32.531 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-28T20:04:32.531 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-28T20:04:32.531 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-28T20:04:32.531 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-28T20:04:32.531 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-28T20:04:32.531 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-28T20:04:32.532 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-28T20:04:32.532 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-28T20:04:32.532 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-28T20:04:32.532 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-28T20:04:32.532 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-28T20:04:32.532 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-28T20:04:32.532 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-28T20:04:32.532 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-28T20:04:32.533 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-28T20:04:32.533 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-28T20:04:32.533 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-28T20:04:32.533 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-28T20:04:32.533 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-28T20:04:32.533 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-28T20:04:32.533 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-28T20:04:32.534 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-06-28T20:04:32.534 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-28T20:04:32.534 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-28T20:04:32.534 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-06-28T20:04:32.534 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-28T20:04:32.534 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-28T20:04:32.534 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-28T20:04:32.534 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-28T20:04:32.535 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-28T20:04:32.535 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-06-28T20:04:32.535 INFO:teuthology.orchestra.run.smithi178.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-28T20:04:32.535 INFO:teuthology.orchestra.run.smithi178.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-28T20:04:32.535 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-28T20:04:32.535 INFO:teuthology.orchestra.run.smithi178.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-28T20:04:32.535 INFO:teuthology.orchestra.run.smithi178.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-28T20:04:32.535 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-28T20:04:32.536 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-28T20:04:32.536 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-28T20:04:32.536 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-28T20:04:32.536 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-28T20:04:32.536 INFO:teuthology.orchestra.run.smithi178.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-06-28T20:04:32.536 INFO:teuthology.orchestra.run.smithi178.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-28T20:04:32.536 INFO:teuthology.orchestra.run.smithi178.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-06-28T20:04:32.537 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-28T20:04:32.537 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-28T20:04:32.537 INFO:teuthology.orchestra.run.smithi178.stdout:Enabling module streams: 2023-06-28T20:04:32.537 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql 10 2023-06-28T20:04:32.537 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:04:32.537 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2023-06-28T20:04:32.537 INFO:teuthology.orchestra.run.smithi178.stdout:=========================================================================================================================== 2023-06-28T20:04:32.538 INFO:teuthology.orchestra.run.smithi178.stdout:Install 128 Packages 2023-06-28T20:04:32.538 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrade 3 Packages 2023-06-28T20:04:32.538 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:04:32.538 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 205 M 2023-06-28T20:04:32.539 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2023-06-28T20:04:33.086 INFO:teuthology.orchestra.run.smithi178.stdout:(1/131): ceph-18.0.0-4628.ga7bcf4ac.el8.x86_64. 53 kB/s | 6.4 kB 00:00 2023-06-28T20:04:33.261 INFO:teuthology.orchestra.run.smithi178.stdout:(2/131): ceph-fuse-18.0.0-4628.ga7bcf4ac.el8.x8 4.7 MB/s | 848 kB 00:00 2023-06-28T20:04:33.311 INFO:teuthology.orchestra.run.smithi178.stdout:(3/131): ceph-immutable-object-cache-18.0.0-462 3.2 MB/s | 165 kB 00:00 2023-06-28T20:04:33.561 INFO:teuthology.orchestra.run.smithi178.stdout:(4/131): ceph-mds-18.0.0-4628.ga7bcf4ac.el8.x86 9.2 MB/s | 2.3 MB 00:00 2023-06-28T20:04:33.770 INFO:teuthology.orchestra.run.smithi178.stdout:(5/131): ceph-mgr-18.0.0-4628.ga7bcf4ac.el8.x86 7.4 MB/s | 1.5 MB 00:00 2023-06-28T20:04:33.845 INFO:teuthology.orchestra.run.smithi178.stdout:(6/131): ceph-base-18.0.0-4628.ga7bcf4ac.el8.x8 6.1 MB/s | 5.4 MB 00:00 2023-06-28T20:04:34.302 INFO:teuthology.orchestra.run.smithi178.stdout:(7/131): ceph-mon-18.0.0-4628.ga7bcf4ac.el8.x86 8.7 MB/s | 4.6 MB 00:00 2023-06-28T20:04:34.420 INFO:teuthology.orchestra.run.smithi043.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-28T20:04:34.421 INFO:teuthology.orchestra.run.smithi043.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-28T20:04:34.435 INFO:teuthology.orchestra.run.smithi043.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-06-28T20:04:34.437 INFO:teuthology.orchestra.run.smithi043.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-06-28T20:04:34.438 INFO:teuthology.orchestra.run.smithi043.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-06-28T20:04:34.439 INFO:teuthology.orchestra.run.smithi043.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-28T20:04:34.441 INFO:teuthology.orchestra.run.smithi043.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-28T20:04:34.444 INFO:teuthology.orchestra.run.smithi043.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-06-28T20:04:34.445 INFO:teuthology.orchestra.run.smithi043.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-06-28T20:04:34.587 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2023-06-28T20:04:34.601 INFO:teuthology.orchestra.run.smithi043.stdout:=========================================================================================================================== 2023-06-28T20:04:34.601 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2023-06-28T20:04:34.601 INFO:teuthology.orchestra.run.smithi043.stdout:=========================================================================================================================== 2023-06-28T20:04:34.601 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2023-06-28T20:04:34.602 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-06-28T20:04:34.602 INFO:teuthology.orchestra.run.smithi043.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-28T20:04:34.602 INFO:teuthology.orchestra.run.smithi043.stdout: ceph x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 6.4 k 2023-06-28T20:04:34.602 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 5.4 M 2023-06-28T20:04:34.602 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 848 k 2023-06-28T20:04:34.602 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 165 k 2023-06-28T20:04:34.602 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 1.5 M 2023-06-28T20:04:34.602 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4628.ga7bcf4ac.el8 ceph-noarch 129 k 2023-06-28T20:04:34.603 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4628.ga7bcf4ac.el8 ceph-noarch 1.7 M 2023-06-28T20:04:34.603 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4628.ga7bcf4ac.el8 ceph-noarch 7.4 M 2023-06-28T20:04:34.603 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook noarch 2:18.0.0-4628.ga7bcf4ac.el8 ceph-noarch 47 k 2023-06-28T20:04:34.603 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 13 M 2023-06-28T20:04:34.603 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 48 M 2023-06-28T20:04:34.603 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume noarch 2:18.0.0-4628.ga7bcf4ac.el8 ceph-noarch 258 k 2023-06-28T20:04:34.603 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm noarch 2:18.0.0-4628.ga7bcf4ac.el8 ceph-noarch 164 k 2023-06-28T20:04:34.604 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-mirror x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 258 k 2023-06-28T20:04:34.604 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-top noarch 2:18.0.0-4628.ga7bcf4ac.el8 ceph-noarch 18 k 2023-06-28T20:04:34.604 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-06-28T20:04:34.604 INFO:teuthology.orchestra.run.smithi043.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-28T20:04:34.604 INFO:teuthology.orchestra.run.smithi043.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-28T20:04:34.604 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 31 k 2023-06-28T20:04:34.605 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2 x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 771 k 2023-06-28T20:04:34.605 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 139 k 2023-06-28T20:04:34.605 INFO:teuthology.orchestra.run.smithi043.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-28T20:04:34.605 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-06-28T20:04:34.605 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-28T20:04:34.605 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-06-28T20:04:34.605 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-06-28T20:04:34.606 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-06-28T20:04:34.606 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 197 k 2023-06-28T20:04:34.606 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 371 k 2023-06-28T20:04:34.606 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 365 k 2023-06-28T20:04:34.606 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 106 k 2023-06-28T20:04:34.606 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 94 k 2023-06-28T20:04:34.606 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 3.4 M 2023-06-28T20:04:34.607 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 179 k 2023-06-28T20:04:34.607 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-06-28T20:04:34.607 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrading: 2023-06-28T20:04:34.607 INFO:teuthology.orchestra.run.smithi043.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-06-28T20:04:34.607 INFO:teuthology.orchestra.run.smithi043.stdout: librados2 x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 3.7 M 2023-06-28T20:04:34.607 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1 x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 3.6 M 2023-06-28T20:04:34.607 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2023-06-28T20:04:34.607 INFO:teuthology.orchestra.run.smithi043.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-28T20:04:34.608 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 22 M 2023-06-28T20:04:34.608 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4628.ga7bcf4ac.el8 ceph-noarch 23 k 2023-06-28T20:04:34.608 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 2.3 M 2023-06-28T20:04:34.608 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4628.ga7bcf4ac.el8 ceph-noarch 240 k 2023-06-28T20:04:34.608 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 4.6 M 2023-06-28T20:04:34.608 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 18 M 2023-06-28T20:04:34.609 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4628.ga7bcf4ac.el8 ceph-noarch 15 k 2023-06-28T20:04:34.609 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 24 k 2023-06-28T20:04:34.609 INFO:teuthology.orchestra.run.smithi043.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-28T20:04:34.609 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-28T20:04:34.609 INFO:teuthology.orchestra.run.smithi043.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-06-28T20:04:34.609 INFO:teuthology.orchestra.run.smithi043.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-28T20:04:34.610 INFO:teuthology.orchestra.run.smithi043.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-06-28T20:04:34.610 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 137 k 2023-06-28T20:04:34.610 INFO:teuthology.orchestra.run.smithi043.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-06-28T20:04:34.610 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-06-28T20:04:34.610 INFO:teuthology.orchestra.run.smithi043.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-28T20:04:34.611 INFO:teuthology.orchestra.run.smithi043.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-06-28T20:04:34.611 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-06-28T20:04:34.611 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-28T20:04:34.611 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1 x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 473 k 2023-06-28T20:04:34.612 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-06-28T20:04:34.612 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2 x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 6.7 M 2023-06-28T20:04:34.612 INFO:teuthology.orchestra.run.smithi043.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-28T20:04:34.613 INFO:teuthology.orchestra.run.smithi043.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-06-28T20:04:34.613 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-28T20:04:34.613 INFO:teuthology.orchestra.run.smithi043.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-28T20:04:34.613 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-28T20:04:34.614 INFO:teuthology.orchestra.run.smithi043.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-28T20:04:34.614 INFO:teuthology.orchestra.run.smithi043.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-28T20:04:34.614 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-06-28T20:04:34.615 INFO:teuthology.orchestra.run.smithi043.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-28T20:04:34.615 INFO:teuthology.orchestra.run.smithi043.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-28T20:04:34.615 INFO:teuthology.orchestra.run.smithi043.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-28T20:04:34.616 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-28T20:04:34.616 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-28T20:04:34.616 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-28T20:04:34.616 INFO:teuthology.orchestra.run.smithi043.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-28T20:04:34.617 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-28T20:04:34.617 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 45 k 2023-06-28T20:04:34.617 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common x86_64 2:18.0.0-4628.ga7bcf4ac.el8 ceph 119 k 2023-06-28T20:04:34.617 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-28T20:04:34.618 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-06-28T20:04:34.618 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-28T20:04:34.618 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-28T20:04:34.619 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-28T20:04:34.619 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-28T20:04:34.619 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-28T20:04:34.619 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-28T20:04:34.619 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-28T20:04:34.620 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-28T20:04:34.620 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-28T20:04:34.620 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-28T20:04:34.620 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-28T20:04:34.620 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-28T20:04:34.621 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-28T20:04:34.621 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-28T20:04:34.621 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-28T20:04:34.621 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-28T20:04:34.621 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-28T20:04:34.621 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-28T20:04:34.621 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-28T20:04:34.622 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-28T20:04:34.622 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-28T20:04:34.622 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-28T20:04:34.622 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-06-28T20:04:34.622 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-28T20:04:34.623 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-28T20:04:34.623 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-06-28T20:04:34.623 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-28T20:04:34.623 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-28T20:04:34.623 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-28T20:04:34.623 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-28T20:04:34.623 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-28T20:04:34.624 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-06-28T20:04:34.624 INFO:teuthology.orchestra.run.smithi043.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-28T20:04:34.624 INFO:teuthology.orchestra.run.smithi043.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-28T20:04:34.624 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-28T20:04:34.624 INFO:teuthology.orchestra.run.smithi043.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-28T20:04:34.624 INFO:teuthology.orchestra.run.smithi043.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-28T20:04:34.624 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-28T20:04:34.625 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-28T20:04:34.625 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-28T20:04:34.625 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-28T20:04:34.625 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-28T20:04:34.625 INFO:teuthology.orchestra.run.smithi043.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-06-28T20:04:34.625 INFO:teuthology.orchestra.run.smithi043.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-28T20:04:34.625 INFO:teuthology.orchestra.run.smithi043.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-06-28T20:04:34.626 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-28T20:04:34.626 INFO:teuthology.orchestra.run.smithi043.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-28T20:04:34.626 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling module streams: 2023-06-28T20:04:34.626 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql 10 2023-06-28T20:04:34.626 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:04:34.626 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2023-06-28T20:04:34.626 INFO:teuthology.orchestra.run.smithi043.stdout:=========================================================================================================================== 2023-06-28T20:04:34.626 INFO:teuthology.orchestra.run.smithi043.stdout:Install 128 Packages 2023-06-28T20:04:34.627 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrade 3 Packages 2023-06-28T20:04:34.627 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:04:34.627 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 205 M 2023-06-28T20:04:34.628 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2023-06-28T20:04:35.213 INFO:teuthology.orchestra.run.smithi043.stdout:(1/131): ceph-18.0.0-4628.ga7bcf4ac.el8.x86_64. 17 kB/s | 6.4 kB 00:00 2023-06-28T20:04:35.363 INFO:teuthology.orchestra.run.smithi178.stdout:(8/131): ceph-radosgw-18.0.0-4628.ga7bcf4ac.el8 13 MB/s | 13 MB 00:01 2023-06-28T20:04:35.462 INFO:teuthology.orchestra.run.smithi178.stdout:(9/131): ceph-selinux-18.0.0-4628.ga7bcf4ac.el8 244 kB/s | 24 kB 00:00 2023-06-28T20:04:36.006 INFO:teuthology.orchestra.run.smithi043.stdout:(2/131): ceph-base-18.0.0-4628.ga7bcf4ac.el8.x8 4.6 MB/s | 5.4 MB 00:01 2023-06-28T20:04:36.031 INFO:teuthology.orchestra.run.smithi043.stdout:(3/131): ceph-fuse-18.0.0-4628.ga7bcf4ac.el8.x8 1.0 MB/s | 848 kB 00:00 2023-06-28T20:04:36.122 INFO:teuthology.orchestra.run.smithi043.stdout:(4/131): ceph-immutable-object-cache-18.0.0-462 1.4 MB/s | 165 kB 00:00 2023-06-28T20:04:36.381 INFO:teuthology.orchestra.run.smithi043.stdout:(5/131): ceph-mgr-18.0.0-4628.ga7bcf4ac.el8.x86 6.0 MB/s | 1.5 MB 00:00 2023-06-28T20:04:36.421 INFO:teuthology.orchestra.run.smithi178.stdout:(10/131): ceph-common-18.0.0-4628.ga7bcf4ac.el8 6.4 MB/s | 22 MB 00:03 2023-06-28T20:04:36.546 INFO:teuthology.orchestra.run.smithi178.stdout:(11/131): cephfs-mirror-18.0.0-4628.ga7bcf4ac.e 2.0 MB/s | 258 kB 00:00 2023-06-28T20:04:36.672 INFO:teuthology.orchestra.run.smithi178.stdout:(12/131): libcephfs-devel-18.0.0-4628.ga7bcf4ac 248 kB/s | 31 kB 00:00 2023-06-28T20:04:36.889 INFO:teuthology.orchestra.run.smithi178.stdout:(13/131): ceph-osd-18.0.0-4628.ga7bcf4ac.el8.x8 5.8 MB/s | 18 MB 00:03 2023-06-28T20:04:36.922 INFO:teuthology.orchestra.run.smithi178.stdout:(14/131): libcephfs2-18.0.0-4628.ga7bcf4ac.el8. 3.0 MB/s | 771 kB 00:00 2023-06-28T20:04:36.989 INFO:teuthology.orchestra.run.smithi178.stdout:(15/131): libcephsqlite-18.0.0-4628.ga7bcf4ac.e 1.3 MB/s | 137 kB 00:00 2023-06-28T20:04:37.015 INFO:teuthology.orchestra.run.smithi043.stdout:(6/131): ceph-mon-18.0.0-4628.ga7bcf4ac.el8.x86 7.3 MB/s | 4.6 MB 00:00 2023-06-28T20:04:37.023 INFO:teuthology.orchestra.run.smithi178.stdout:(16/131): librados-devel-18.0.0-4628.ga7bcf4ac. 1.4 MB/s | 139 kB 00:00 2023-06-28T20:04:37.148 INFO:teuthology.orchestra.run.smithi178.stdout:(17/131): libradosstriper1-18.0.0-4628.ga7bcf4a 2.9 MB/s | 473 kB 00:00 2023-06-28T20:04:37.173 INFO:teuthology.orchestra.run.smithi043.stdout:(7/131): ceph-mds-18.0.0-4628.ga7bcf4ac.el8.x86 2.0 MB/s | 2.3 MB 00:01 2023-06-28T20:04:37.249 INFO:teuthology.orchestra.run.smithi178.stdout:(18/131): python3-ceph-argparse-18.0.0-4628.ga7 447 kB/s | 45 kB 00:00 2023-06-28T20:04:37.349 INFO:teuthology.orchestra.run.smithi178.stdout:(19/131): python3-ceph-common-18.0.0-4628.ga7bc 1.2 MB/s | 119 kB 00:00 2023-06-28T20:04:37.466 INFO:teuthology.orchestra.run.smithi178.stdout:(20/131): python3-cephfs-18.0.0-4628.ga7bcf4ac. 1.7 MB/s | 197 kB 00:00 2023-06-28T20:04:37.591 INFO:teuthology.orchestra.run.smithi178.stdout:(21/131): python3-rados-18.0.0-4628.ga7bcf4ac.e 2.9 MB/s | 371 kB 00:00 2023-06-28T20:04:37.708 INFO:teuthology.orchestra.run.smithi178.stdout:(22/131): python3-rbd-18.0.0-4628.ga7bcf4ac.el8 3.1 MB/s | 365 kB 00:00 2023-06-28T20:04:37.808 INFO:teuthology.orchestra.run.smithi178.stdout:(23/131): python3-rgw-18.0.0-4628.ga7bcf4ac.el8 1.0 MB/s | 106 kB 00:00 2023-06-28T20:04:37.900 INFO:teuthology.orchestra.run.smithi178.stdout:(24/131): rbd-fuse-18.0.0-4628.ga7bcf4ac.el8.x8 1.0 MB/s | 94 kB 00:00 2023-06-28T20:04:38.742 INFO:teuthology.orchestra.run.smithi178.stdout:(25/131): rbd-mirror-18.0.0-4628.ga7bcf4ac.el8. 4.1 MB/s | 3.4 MB 00:00 2023-06-28T20:04:39.026 INFO:teuthology.orchestra.run.smithi178.stdout:(26/131): librgw2-18.0.0-4628.ga7bcf4ac.el8.x86 3.4 MB/s | 6.7 MB 00:02 2023-06-28T20:04:39.043 INFO:teuthology.orchestra.run.smithi178.stdout:(27/131): rbd-nbd-18.0.0-4628.ga7bcf4ac.el8.x86 597 kB/s | 179 kB 00:00 2023-06-28T20:04:39.118 INFO:teuthology.orchestra.run.smithi178.stdout:(28/131): ceph-grafana-dashboards-18.0.0-4628.g 251 kB/s | 23 kB 00:00 2023-06-28T20:04:39.152 INFO:teuthology.orchestra.run.smithi178.stdout:(29/131): ceph-mgr-cephadm-18.0.0-4628.ga7bcf4a 1.2 MB/s | 129 kB 00:00 2023-06-28T20:04:39.682 INFO:teuthology.orchestra.run.smithi043.stdout:(8/131): ceph-radosgw-18.0.0-4628.ga7bcf4ac.el8 5.4 MB/s | 13 MB 00:02 2023-06-28T20:04:39.719 INFO:teuthology.orchestra.run.smithi178.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4628.ga7bcf 2.9 MB/s | 1.7 MB 00:00 2023-06-28T20:04:39.757 INFO:teuthology.orchestra.run.smithi043.stdout:(9/131): ceph-selinux-18.0.0-4628.ga7bcf4ac.el8 325 kB/s | 24 kB 00:00 2023-06-28T20:04:39.819 INFO:teuthology.orchestra.run.smithi178.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4628.ga7 2.4 MB/s | 240 kB 00:00 2023-06-28T20:04:39.911 INFO:teuthology.orchestra.run.smithi178.stdout:(32/131): ceph-mgr-rook-18.0.0-4628.ga7bcf4ac.e 518 kB/s | 47 kB 00:00 2023-06-28T20:04:39.986 INFO:teuthology.orchestra.run.smithi178.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4628.ga 195 kB/s | 15 kB 00:00 2023-06-28T20:04:40.016 INFO:teuthology.orchestra.run.smithi043.stdout:(10/131): ceph-osd-18.0.0-4628.ga7bcf4ac.el8.x8 5.9 MB/s | 18 MB 00:03 2023-06-28T20:04:40.078 INFO:teuthology.orchestra.run.smithi178.stdout:(34/131): ceph-volume-18.0.0-4628.ga7bcf4ac.el8 2.8 MB/s | 258 kB 00:00 2023-06-28T20:04:40.108 INFO:teuthology.orchestra.run.smithi043.stdout:(11/131): cephfs-mirror-18.0.0-4628.ga7bcf4ac.e 2.8 MB/s | 258 kB 00:00 2023-06-28T20:04:40.180 INFO:teuthology.orchestra.run.smithi178.stdout:(35/131): cephadm-18.0.0-4628.ga7bcf4ac.el8.noa 1.6 MB/s | 164 kB 00:00 2023-06-28T20:04:40.197 INFO:teuthology.orchestra.run.smithi043.stdout:(12/131): libcephfs-devel-18.0.0-4628.ga7bcf4ac 348 kB/s | 31 kB 00:00 2023-06-28T20:04:40.254 INFO:teuthology.orchestra.run.smithi178.stdout:(36/131): cephfs-top-18.0.0-4628.ga7bcf4ac.el8. 246 kB/s | 18 kB 00:00 2023-06-28T20:04:40.358 INFO:teuthology.orchestra.run.smithi043.stdout:(13/131): libcephfs2-18.0.0-4628.ga7bcf4ac.el8. 4.7 MB/s | 771 kB 00:00 2023-06-28T20:04:40.433 INFO:teuthology.orchestra.run.smithi043.stdout:(14/131): libcephsqlite-18.0.0-4628.ga7bcf4ac.e 1.8 MB/s | 137 kB 00:00 2023-06-28T20:04:40.500 INFO:teuthology.orchestra.run.smithi043.stdout:(15/131): librados-devel-18.0.0-4628.ga7bcf4ac. 2.0 MB/s | 139 kB 00:00 2023-06-28T20:04:40.554 INFO:teuthology.orchestra.run.smithi178.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 5.3 MB/s | 7.4 MB 00:01 2023-06-28T20:04:40.601 INFO:teuthology.orchestra.run.smithi043.stdout:(16/131): libradosstriper1-18.0.0-4628.ga7bcf4a 4.6 MB/s | 473 kB 00:00 2023-06-28T20:04:41.046 INFO:teuthology.orchestra.run.smithi178.stdout:(38/131): ceph-test-18.0.0-4628.ga7bcf4ac.el8.x 8.5 MB/s | 48 MB 00:05 2023-06-28T20:04:41.163 INFO:teuthology.orchestra.run.smithi178.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 337 kB/s | 306 kB 00:00 2023-06-28T20:04:41.243 INFO:teuthology.orchestra.run.smithi043.stdout:(17/131): librgw2-18.0.0-4628.ga7bcf4ac.el8.x86 11 MB/s | 6.7 MB 00:00 2023-06-28T20:04:41.301 INFO:teuthology.orchestra.run.smithi043.stdout:(18/131): python3-ceph-argparse-18.0.0-4628.ga7 771 kB/s | 45 kB 00:00 2023-06-28T20:04:41.368 INFO:teuthology.orchestra.run.smithi043.stdout:(19/131): python3-ceph-common-18.0.0-4628.ga7bc 1.7 MB/s | 119 kB 00:00 2023-06-28T20:04:41.371 INFO:teuthology.orchestra.run.smithi178.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.3 MB/s | 476 kB 00:00 2023-06-28T20:04:41.435 INFO:teuthology.orchestra.run.smithi043.stdout:(20/131): python3-cephfs-18.0.0-4628.ga7bcf4ac. 2.9 MB/s | 197 kB 00:00 2023-06-28T20:04:41.438 INFO:teuthology.orchestra.run.smithi178.stdout:(41/131): liboath-2.6.2-3.el8.x86_64.rpm 66 kB/s | 59 kB 00:00 2023-06-28T20:04:41.481 INFO:teuthology.orchestra.run.smithi178.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 400 kB/s | 44 kB 00:00 2023-06-28T20:04:41.502 INFO:teuthology.orchestra.run.smithi043.stdout:(21/131): python3-rados-18.0.0-4628.ga7bcf4ac.e 5.4 MB/s | 371 kB 00:00 2023-06-28T20:04:41.539 INFO:teuthology.orchestra.run.smithi178.stdout:(43/131): libunwind-1.3.1-3.el8.x86_64.rpm 154 kB/s | 75 kB 00:00 2023-06-28T20:04:41.569 INFO:teuthology.orchestra.run.smithi043.stdout:(22/131): python3-rbd-18.0.0-4628.ga7bcf4ac.el8 5.4 MB/s | 365 kB 00:00 2023-06-28T20:04:41.589 INFO:teuthology.orchestra.run.smithi178.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 304 kB/s | 33 kB 00:00 2023-06-28T20:04:41.611 INFO:teuthology.orchestra.run.smithi043.stdout:(23/131): python3-rgw-18.0.0-4628.ga7bcf4ac.el8 2.5 MB/s | 106 kB 00:00 2023-06-28T20:04:41.648 INFO:teuthology.orchestra.run.smithi178.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 150 kB/s | 16 kB 00:00 2023-06-28T20:04:41.653 INFO:teuthology.orchestra.run.smithi043.stdout:(24/131): rbd-fuse-18.0.0-4628.ga7bcf4ac.el8.x8 2.2 MB/s | 94 kB 00:00 2023-06-28T20:04:41.673 INFO:teuthology.orchestra.run.smithi178.stdout:(46/131): python3-beautifulsoup4-4.6.3-2.el8.1. 789 kB/s | 185 kB 00:00 2023-06-28T20:04:41.698 INFO:teuthology.orchestra.run.smithi178.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.5 MB/s | 173 kB 00:00 2023-06-28T20:04:41.782 INFO:teuthology.orchestra.run.smithi178.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 750 kB/s | 81 kB 00:00 2023-06-28T20:04:41.808 INFO:teuthology.orchestra.run.smithi178.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 102 kB/s | 11 kB 00:00 2023-06-28T20:04:41.915 INFO:teuthology.orchestra.run.smithi178.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 132 kB/s | 18 kB 00:00 2023-06-28T20:04:41.950 INFO:teuthology.orchestra.run.smithi178.stdout:(51/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.2 MB/s | 384 kB 00:00 2023-06-28T20:04:42.003 INFO:teuthology.orchestra.run.smithi043.stdout:(25/131): ceph-common-18.0.0-4628.ga7bcf4ac.el8 3.1 MB/s | 22 MB 00:07 2023-06-28T20:04:42.017 INFO:teuthology.orchestra.run.smithi178.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 5.6 MB/s | 1.2 MB 00:00 2023-06-28T20:04:42.042 INFO:teuthology.orchestra.run.smithi178.stdout:(53/131): python3-logutils-0.3.5-11.el8.noarch. 385 kB/s | 49 kB 00:00 2023-06-28T20:04:42.067 INFO:teuthology.orchestra.run.smithi178.stdout:(54/131): python3-more-itertools-7.2.0-3.el8.no 500 kB/s | 59 kB 00:00 2023-06-28T20:04:42.112 INFO:teuthology.orchestra.run.smithi043.stdout:(26/131): rbd-mirror-18.0.0-4628.ga7bcf4ac.el8. 7.5 MB/s | 3.4 MB 00:00 2023-06-28T20:04:42.117 INFO:teuthology.orchestra.run.smithi178.stdout:(55/131): python3-natsort-7.1.1-2.el8.noarch.rp 600 kB/s | 60 kB 00:00 2023-06-28T20:04:42.137 INFO:teuthology.orchestra.run.smithi043.stdout:(27/131): rbd-nbd-18.0.0-4628.ga7bcf4ac.el8.x86 1.3 MB/s | 179 kB 00:00 2023-06-28T20:04:42.154 INFO:teuthology.orchestra.run.smithi043.stdout:(28/131): ceph-grafana-dashboards-18.0.0-4628.g 551 kB/s | 23 kB 00:00 2023-06-28T20:04:42.176 INFO:teuthology.orchestra.run.smithi178.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 143 kB/s | 16 kB 00:00 2023-06-28T20:04:42.196 INFO:teuthology.orchestra.run.smithi043.stdout:(29/131): ceph-mgr-cephadm-18.0.0-4628.ga7bcf4a 2.1 MB/s | 129 kB 00:00 2023-06-28T20:04:42.218 INFO:teuthology.orchestra.run.smithi178.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 336 kB/s | 34 kB 00:00 2023-06-28T20:04:42.243 INFO:teuthology.orchestra.run.smithi178.stdout:(58/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.4 MB/s | 283 kB 00:00 2023-06-28T20:04:42.285 INFO:teuthology.orchestra.run.smithi178.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.8 MB/s | 196 kB 00:00 2023-06-28T20:04:42.319 INFO:teuthology.orchestra.run.smithi178.stdout:(60/131): python3-rsa-4.8-1.el8.noarch.rpm 612 kB/s | 61 kB 00:00 2023-06-28T20:04:42.344 INFO:teuthology.orchestra.run.smithi178.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 186 kB/s | 19 kB 00:00 2023-06-28T20:04:42.371 INFO:teuthology.orchestra.run.smithi043.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4628.ga7bcf 7.9 MB/s | 1.7 MB 00:00 2023-06-28T20:04:42.386 INFO:teuthology.orchestra.run.smithi178.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 238 kB/s | 24 kB 00:00 2023-06-28T20:04:42.420 INFO:teuthology.orchestra.run.smithi178.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 291 kB/s | 29 kB 00:00 2023-06-28T20:04:42.421 INFO:teuthology.orchestra.run.smithi043.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4628.ga7 4.7 MB/s | 240 kB 00:00 2023-06-28T20:04:42.445 INFO:teuthology.orchestra.run.smithi178.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 273 kB/s | 27 kB 00:00 2023-06-28T20:04:42.455 INFO:teuthology.orchestra.run.smithi043.stdout:(32/131): ceph-mgr-rook-18.0.0-4628.ga7bcf4ac.e 1.4 MB/s | 47 kB 00:00 2023-06-28T20:04:42.489 INFO:teuthology.orchestra.run.smithi043.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4628.ga 438 kB/s | 15 kB 00:00 2023-06-28T20:04:42.512 INFO:teuthology.orchestra.run.smithi178.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 2.0 MB/s | 255 kB 00:00 2023-06-28T20:04:42.537 INFO:teuthology.orchestra.run.smithi178.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.1 MB/s | 251 kB 00:00 2023-06-28T20:04:42.539 INFO:teuthology.orchestra.run.smithi043.stdout:(34/131): ceph-volume-18.0.0-4628.ga7bcf4ac.el8 5.1 MB/s | 258 kB 00:00 2023-06-28T20:04:42.554 INFO:teuthology.orchestra.run.smithi178.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 563 kB/s | 61 kB 00:00 2023-06-28T20:04:42.598 INFO:teuthology.orchestra.run.smithi043.stdout:(35/131): cephadm-18.0.0-4628.ga7bcf4ac.el8.noa 2.8 MB/s | 164 kB 00:00 2023-06-28T20:04:42.637 INFO:teuthology.orchestra.run.smithi178.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 681 kB/s | 85 kB 00:00 2023-06-28T20:04:42.640 INFO:teuthology.orchestra.run.smithi043.stdout:(36/131): cephfs-top-18.0.0-4628.ga7bcf4ac.el8. 430 kB/s | 18 kB 00:00 2023-06-28T20:04:42.654 INFO:teuthology.orchestra.run.smithi178.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 196 kB/s | 23 kB 00:00 2023-06-28T20:04:42.680 INFO:teuthology.orchestra.run.smithi178.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 70 kB 00:00 2023-06-28T20:04:42.730 INFO:teuthology.orchestra.run.smithi178.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 623 kB/s | 47 kB 00:00 2023-06-28T20:04:42.756 INFO:teuthology.orchestra.run.smithi178.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.8 MB/s | 200 kB 00:00 2023-06-28T20:04:42.781 INFO:teuthology.orchestra.run.smithi178.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 845 kB/s | 43 kB 00:00 2023-06-28T20:04:42.806 INFO:teuthology.orchestra.run.smithi178.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-06-28T20:04:42.831 INFO:teuthology.orchestra.run.smithi178.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 963 kB/s | 48 kB 00:00 2023-06-28T20:04:42.857 INFO:teuthology.orchestra.run.smithi178.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-06-28T20:04:42.882 INFO:teuthology.orchestra.run.smithi043.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-06-28T20:04:42.882 INFO:teuthology.orchestra.run.smithi178.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 850 kB/s | 43 kB 00:00 2023-06-28T20:04:42.907 INFO:teuthology.orchestra.run.smithi178.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-06-28T20:04:42.940 INFO:teuthology.orchestra.run.smithi043.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.0 MB/s | 59 kB 00:00 2023-06-28T20:04:42.949 INFO:teuthology.orchestra.run.smithi178.stdout:(79/131): thrift-0.13.0-2.el8.x86_64.rpm 4.4 MB/s | 1.7 MB 00:00 2023-06-28T20:04:42.974 INFO:teuthology.orchestra.run.smithi178.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.4 MB/s | 229 kB 00:00 2023-06-28T20:04:42.999 INFO:teuthology.orchestra.run.smithi043.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.3 MB/s | 75 kB 00:00 2023-06-28T20:04:43.008 INFO:teuthology.orchestra.run.smithi178.stdout:(81/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 5.4 MB/s | 560 kB 00:00 2023-06-28T20:04:43.033 INFO:teuthology.orchestra.run.smithi178.stdout:(82/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 223 kB/s | 19 kB 00:00 2023-06-28T20:04:43.092 INFO:teuthology.orchestra.run.smithi178.stdout:(83/131): python3-oauthlib-2.1.0-1.el8.noarch.r 1.4 MB/s | 155 kB 00:00 2023-06-28T20:04:43.099 INFO:teuthology.orchestra.run.smithi043.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 4.7 MB/s | 476 kB 00:00 2023-06-28T20:04:43.117 INFO:teuthology.orchestra.run.smithi178.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 214 kB/s | 18 kB 00:00 2023-06-28T20:04:43.151 INFO:teuthology.orchestra.run.smithi178.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 3.8 MB/s | 559 kB 00:00 2023-06-28T20:04:43.158 INFO:teuthology.orchestra.run.smithi043.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 750 kB/s | 44 kB 00:00 2023-06-28T20:04:43.176 INFO:teuthology.orchestra.run.smithi178.stdout:(86/131): pcre2-devel-10.32-2.el8.x86_64.rpm 7.1 MB/s | 605 kB 00:00 2023-06-28T20:04:43.201 INFO:teuthology.orchestra.run.smithi178.stdout:(87/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.8 MB/s | 238 kB 00:00 2023-06-28T20:04:43.227 INFO:teuthology.orchestra.run.smithi178.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-06-28T20:04:43.241 INFO:teuthology.orchestra.run.smithi043.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.2 MB/s | 185 kB 00:00 2023-06-28T20:04:43.252 INFO:teuthology.orchestra.run.smithi178.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-06-28T20:04:43.277 INFO:teuthology.orchestra.run.smithi178.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-06-28T20:04:43.300 INFO:teuthology.orchestra.run.smithi043.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 565 kB/s | 33 kB 00:00 2023-06-28T20:04:43.302 INFO:teuthology.orchestra.run.smithi178.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-06-28T20:04:43.327 INFO:teuthology.orchestra.run.smithi178.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-06-28T20:04:43.353 INFO:teuthology.orchestra.run.smithi178.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 769 kB/s | 39 kB 00:00 2023-06-28T20:04:43.359 INFO:teuthology.orchestra.run.smithi043.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 280 kB/s | 16 kB 00:00 2023-06-28T20:04:43.386 INFO:teuthology.orchestra.run.smithi178.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-06-28T20:04:43.412 INFO:teuthology.orchestra.run.smithi178.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-06-28T20:04:43.417 INFO:teuthology.orchestra.run.smithi043.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.9 MB/s | 173 kB 00:00 2023-06-28T20:04:43.437 INFO:teuthology.orchestra.run.smithi178.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-06-28T20:04:43.484 INFO:teuthology.orchestra.run.smithi043.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 5.6 MB/s | 384 kB 00:00 2023-06-28T20:04:43.495 INFO:teuthology.orchestra.run.smithi178.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 6.7 MB/s | 2.3 MB 00:00 2023-06-28T20:04:43.521 INFO:teuthology.orchestra.run.smithi178.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.3 MB/s | 110 kB 00:00 2023-06-28T20:04:43.546 INFO:teuthology.orchestra.run.smithi178.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-06-28T20:04:43.571 INFO:teuthology.orchestra.run.smithi178.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-06-28T20:04:43.605 INFO:teuthology.orchestra.run.smithi178.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-06-28T20:04:43.655 INFO:teuthology.orchestra.run.smithi178.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 465 kB/s | 39 kB 00:00 2023-06-28T20:04:43.697 INFO:teuthology.orchestra.run.smithi178.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 7.3 MB/s | 688 kB 00:00 2023-06-28T20:04:43.739 INFO:teuthology.orchestra.run.smithi178.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 6.3 MB/s | 538 kB 00:00 2023-06-28T20:04:43.765 INFO:teuthology.orchestra.run.smithi178.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-06-28T20:04:43.790 INFO:teuthology.orchestra.run.smithi178.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-06-28T20:04:43.832 INFO:teuthology.orchestra.run.smithi178.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-06-28T20:04:43.874 INFO:teuthology.orchestra.run.smithi178.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 641 kB/s | 27 kB 00:00 2023-06-28T20:04:43.924 INFO:teuthology.orchestra.run.smithi178.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-06-28T20:04:44.135 INFO:teuthology.orchestra.run.smithi043.stdout:(47/131): ceph-test-18.0.0-4628.ga7bcf4ac.el8.x 11 MB/s | 48 MB 00:04 2023-06-28T20:04:44.160 INFO:teuthology.orchestra.run.smithi043.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 120 kB/s | 81 kB 00:00 2023-06-28T20:04:44.191 INFO:teuthology.orchestra.run.smithi178.stdout:(110/131): python3-babel-2.5.1-7.el8.noarch.rpm 12 MB/s | 4.8 MB 00:00 2023-06-28T20:04:44.241 INFO:teuthology.orchestra.run.smithi178.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 139 kB/s | 44 kB 00:00 2023-06-28T20:04:44.244 INFO:teuthology.orchestra.run.smithi043.stdout:(49/131): ceph-mgr-diskprediction-local-18.0.0- 3.6 MB/s | 7.4 MB 00:02 2023-06-28T20:04:44.269 INFO:teuthology.orchestra.run.smithi043.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2023-06-28T20:04:44.294 INFO:teuthology.orchestra.run.smithi043.stdout:(51/131): python3-jaraco-6.2-6.el8.noarch.rpm 70 kB/s | 11 kB 00:00 2023-06-28T20:04:44.300 INFO:teuthology.orchestra.run.smithi178.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 7.7 MB/s | 457 kB 00:00 2023-06-28T20:04:44.328 INFO:teuthology.orchestra.run.smithi043.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 830 kB/s | 49 kB 00:00 2023-06-28T20:04:44.375 INFO:teuthology.orchestra.run.smithi178.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 20 MB/s | 3.7 MB 00:00 2023-06-28T20:04:44.395 INFO:teuthology.orchestra.run.smithi043.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 908 kB/s | 60 kB 00:00 2023-06-28T20:04:44.400 INFO:teuthology.orchestra.run.smithi178.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.2 MB/s | 225 kB 00:00 2023-06-28T20:04:44.420 INFO:teuthology.orchestra.run.smithi043.stdout:(54/131): python3-more-itertools-7.2.0-3.el8.no 467 kB/s | 59 kB 00:00 2023-06-28T20:04:44.426 INFO:teuthology.orchestra.run.smithi178.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2023-06-28T20:04:44.451 INFO:teuthology.orchestra.run.smithi178.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-06-28T20:04:44.476 INFO:teuthology.orchestra.run.smithi178.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 398 kB/s | 20 kB 00:00 2023-06-28T20:04:44.479 INFO:teuthology.orchestra.run.smithi043.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 268 kB/s | 16 kB 00:00 2023-06-28T20:04:44.502 INFO:teuthology.orchestra.run.smithi178.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-06-28T20:04:44.504 INFO:teuthology.orchestra.run.smithi043.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2023-06-28T20:04:44.538 INFO:teuthology.orchestra.run.smithi043.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 574 kB/s | 34 kB 00:00 2023-06-28T20:04:44.580 INFO:teuthology.orchestra.run.smithi043.stdout:(58/131): python3-kubernetes-11.0.0-6.el8.noarc 3.5 MB/s | 1.2 MB 00:00 2023-06-28T20:04:44.605 INFO:teuthology.orchestra.run.smithi043.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 918 kB/s | 61 kB 00:00 2023-06-28T20:04:44.639 INFO:teuthology.orchestra.run.smithi043.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 319 kB/s | 19 kB 00:00 2023-06-28T20:04:44.644 INFO:teuthology.orchestra.run.smithi178.stdout:(119/131): python3-scipy-1.0.0-21.module+el8.5. 11 MB/s | 14 MB 00:01 2023-06-28T20:04:44.664 INFO:teuthology.orchestra.run.smithi043.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 409 kB/s | 24 kB 00:00 2023-06-28T20:04:44.677 INFO:teuthology.orchestra.run.smithi178.stdout:(120/131): libtool-2.4.6-25.el8.x86_64.rpm 3.5 MB/s | 709 kB 00:00 2023-06-28T20:04:44.689 INFO:teuthology.orchestra.run.smithi043.stdout:(62/131): python3-routes-2.4.1-12.el8.noarch.rp 1.0 MB/s | 196 kB 00:00 2023-06-28T20:04:44.702 INFO:teuthology.orchestra.run.smithi178.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 3.3 MB/s | 198 kB 00:00 2023-06-28T20:04:44.715 INFO:teuthology.orchestra.run.smithi043.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 387 kB/s | 29 kB 00:00 2023-06-28T20:04:44.740 INFO:teuthology.orchestra.run.smithi043.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 365 kB/s | 27 kB 00:00 2023-06-28T20:04:44.761 INFO:teuthology.orchestra.run.smithi178.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 3.4 MB/s | 202 kB 00:00 2023-06-28T20:04:44.773 INFO:teuthology.orchestra.run.smithi043.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 4.2 MB/s | 251 kB 00:00 2023-06-28T20:04:44.799 INFO:teuthology.orchestra.run.smithi043.stdout:(66/131): python3-waitress-1.4.3-1.el8.noarch.r 2.3 MB/s | 255 kB 00:00 2023-06-28T20:04:44.824 INFO:teuthology.orchestra.run.smithi043.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 733 kB/s | 61 kB 00:00 2023-06-28T20:04:44.841 INFO:teuthology.orchestra.run.smithi043.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 1.2 MB/s | 85 kB 00:00 2023-06-28T20:04:44.862 INFO:teuthology.orchestra.run.smithi178.stdout:(123/131): openblas-threads-0.3.15-3.el8.x86_64 14 MB/s | 4.9 MB 00:00 2023-06-28T20:04:44.866 INFO:teuthology.orchestra.run.smithi043.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-06-28T20:04:44.891 INFO:teuthology.orchestra.run.smithi043.stdout:(70/131): python3-zc-lockfile-2.0-2.el8.noarch. 343 kB/s | 23 kB 00:00 2023-06-28T20:04:44.895 INFO:teuthology.orchestra.run.smithi178.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2023-06-28T20:04:44.962 INFO:teuthology.orchestra.run.smithi178.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-06-28T20:04:44.966 INFO:teuthology.orchestra.run.smithi043.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.6 MB/s | 200 kB 00:00 2023-06-28T20:04:44.992 INFO:teuthology.orchestra.run.smithi043.stdout:(72/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 373 kB/s | 47 kB 00:00 2023-06-28T20:04:45.004 INFO:teuthology.orchestra.run.smithi178.stdout:(126/131): postgresql-contrib-10.21-2.module+el 7.3 MB/s | 808 kB 00:00 2023-06-28T20:04:45.026 INFO:teuthology.orchestra.run.smithi043.stdout:(73/131): thrift-0.13.0-2.el8.x86_64.rpm 8.6 MB/s | 1.7 MB 00:00 2023-06-28T20:04:45.051 INFO:teuthology.orchestra.run.smithi043.stdout:(74/131): python3-jwt-1.6.1-2.el8.noarch.rpm 508 kB/s | 43 kB 00:00 2023-06-28T20:04:45.062 INFO:teuthology.orchestra.run.smithi178.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 15 MB/s | 1.5 MB 00:00 2023-06-28T20:04:45.085 INFO:teuthology.orchestra.run.smithi043.stdout:(75/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.3 MB/s | 220 kB 00:00 2023-06-28T20:04:45.110 INFO:teuthology.orchestra.run.smithi043.stdout:(76/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 573 kB/s | 48 kB 00:00 2023-06-28T20:04:45.127 INFO:teuthology.orchestra.run.smithi043.stdout:(77/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.3 MB/s | 87 kB 00:00 2023-06-28T20:04:45.144 INFO:teuthology.orchestra.run.smithi043.stdout:(78/131): python3-requests-oauthlib-1.0.0-1.el8 723 kB/s | 43 kB 00:00 2023-06-28T20:04:45.146 INFO:teuthology.orchestra.run.smithi178.stdout:(128/131): postgresql-server-10.21-2.module+el8 18 MB/s | 5.1 MB 00:00 2023-06-28T20:04:45.161 INFO:teuthology.orchestra.run.smithi043.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-06-28T20:04:45.186 INFO:teuthology.orchestra.run.smithi043.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-06-28T20:04:45.203 INFO:teuthology.orchestra.run.smithi043.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 446 kB/s | 19 kB 00:00 2023-06-28T20:04:45.213 INFO:teuthology.orchestra.run.smithi178.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 12 MB/s | 798 kB 00:00 2023-06-28T20:04:45.220 INFO:teuthology.orchestra.run.smithi043.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 4.5 MB/s | 155 kB 00:00 2023-06-28T20:04:45.254 INFO:teuthology.orchestra.run.smithi043.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 5.0 MB/s | 560 kB 00:00 2023-06-28T20:04:45.279 INFO:teuthology.orchestra.run.smithi043.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 306 kB/s | 18 kB 00:00 2023-06-28T20:04:45.313 INFO:teuthology.orchestra.run.smithi043.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 5.0 MB/s | 559 kB 00:00 2023-06-28T20:04:45.338 INFO:teuthology.orchestra.run.smithi043.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-06-28T20:04:45.388 INFO:teuthology.orchestra.run.smithi043.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.4 MB/s | 605 kB 00:00 2023-06-28T20:04:45.413 INFO:teuthology.orchestra.run.smithi178.stdout:(130/131): librados2-18.0.0-4628.ga7bcf4ac.el8. 8.9 MB/s | 3.7 MB 00:00 2023-06-28T20:04:45.414 INFO:teuthology.orchestra.run.smithi043.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 962 kB/s | 72 kB 00:00 2023-06-28T20:04:45.439 INFO:teuthology.orchestra.run.smithi043.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-06-28T20:04:45.464 INFO:teuthology.orchestra.run.smithi043.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-06-28T20:04:45.489 INFO:teuthology.orchestra.run.smithi043.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-06-28T20:04:45.514 INFO:teuthology.orchestra.run.smithi043.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-06-28T20:04:45.540 INFO:teuthology.orchestra.run.smithi043.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 772 kB/s | 39 kB 00:00 2023-06-28T20:04:45.573 INFO:teuthology.orchestra.run.smithi043.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-06-28T20:04:45.599 INFO:teuthology.orchestra.run.smithi043.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-06-28T20:04:45.624 INFO:teuthology.orchestra.run.smithi043.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-06-28T20:04:45.666 INFO:teuthology.orchestra.run.smithi043.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-06-28T20:04:45.708 INFO:teuthology.orchestra.run.smithi043.stdout:(98/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 5.9 MB/s | 2.3 MB 00:00 2023-06-28T20:04:45.733 INFO:teuthology.orchestra.run.smithi043.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 812 kB/s | 54 kB 00:00 2023-06-28T20:04:45.767 INFO:teuthology.orchestra.run.smithi043.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.3 MB/s | 259 kB 00:00 2023-06-28T20:04:45.801 INFO:teuthology.orchestra.run.smithi043.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-06-28T20:04:45.826 INFO:teuthology.orchestra.run.smithi043.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 664 kB/s | 39 kB 00:00 2023-06-28T20:04:45.876 INFO:teuthology.orchestra.run.smithi043.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 9.0 MB/s | 688 kB 00:00 2023-06-28T20:04:45.910 INFO:teuthology.orchestra.run.smithi043.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 6.3 MB/s | 538 kB 00:00 2023-06-28T20:04:45.935 INFO:teuthology.orchestra.run.smithi043.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-06-28T20:04:45.960 INFO:teuthology.orchestra.run.smithi043.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-06-28T20:04:46.002 INFO:teuthology.orchestra.run.smithi043.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-06-28T20:04:46.044 INFO:teuthology.orchestra.run.smithi043.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 637 kB/s | 27 kB 00:00 2023-06-28T20:04:46.086 INFO:teuthology.orchestra.run.smithi043.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-06-28T20:04:46.128 INFO:teuthology.orchestra.run.smithi043.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.0 MB/s | 44 kB 00:00 2023-06-28T20:04:46.220 INFO:teuthology.orchestra.run.smithi043.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2023-06-28T20:04:46.221 INFO:teuthology.orchestra.run.smithi178.stdout:(131/131): librbd1-18.0.0-4628.ga7bcf4ac.el8.x8 3.1 MB/s | 3.6 MB 00:01 2023-06-28T20:04:46.227 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2023-06-28T20:04:46.227 INFO:teuthology.orchestra.run.smithi178.stdout:Total 15 MB/s | 205 MB 00:13 2023-06-28T20:04:46.270 INFO:teuthology.orchestra.run.smithi043.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 9.0 MB/s | 457 kB 00:00 2023-06-28T20:04:46.312 INFO:teuthology.orchestra.run.smithi043.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-06-28T20:04:46.371 INFO:teuthology.orchestra.run.smithi043.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 15 MB/s | 3.7 MB 00:00 2023-06-28T20:04:46.521 INFO:teuthology.orchestra.run.smithi043.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 307 kB/s | 64 kB 00:00 2023-06-28T20:04:46.546 INFO:teuthology.orchestra.run.smithi043.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 723 kB/s | 126 kB 00:00 2023-06-28T20:04:46.572 INFO:teuthology.orchestra.run.smithi043.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 397 kB/s | 20 kB 00:00 2023-06-28T20:04:46.597 INFO:teuthology.orchestra.run.smithi043.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-06-28T20:04:46.664 INFO:teuthology.orchestra.run.smithi043.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 7.5 MB/s | 709 kB 00:00 2023-06-28T20:04:46.706 INFO:teuthology.orchestra.run.smithi043.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2023-06-28T20:04:46.839 INFO:teuthology.orchestra.run.smithi043.stdout:(121/131): python3-scipy-1.0.0-21.module+el8.5. 11 MB/s | 14 MB 00:01 2023-06-28T20:04:46.881 INFO:teuthology.orchestra.run.smithi043.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-06-28T20:04:46.923 INFO:teuthology.orchestra.run.smithi043.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-06-28T20:04:47.007 INFO:teuthology.orchestra.run.smithi043.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 12 MB/s | 4.9 MB 00:00 2023-06-28T20:04:47.082 INFO:teuthology.orchestra.run.smithi043.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 13 MB/s | 4.7 MB 00:00 2023-06-28T20:04:47.088 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2023-06-28T20:04:47.124 INFO:teuthology.orchestra.run.smithi043.stdout:(126/131): postgresql-contrib-10.21-2.module+el 6.7 MB/s | 808 kB 00:00 2023-06-28T20:04:47.183 INFO:teuthology.orchestra.run.smithi043.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 15 MB/s | 1.5 MB 00:00 2023-06-28T20:04:47.208 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2023-06-28T20:04:47.208 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2023-06-28T20:04:47.266 INFO:teuthology.orchestra.run.smithi043.stdout:(128/131): postgresql-server-10.21-2.module+el8 15 MB/s | 5.1 MB 00:00 2023-06-28T20:04:47.325 INFO:teuthology.orchestra.run.smithi043.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-06-28T20:04:47.617 INFO:teuthology.orchestra.run.smithi043.stdout:(130/131): librbd1-18.0.0-4628.ga7bcf4ac.el8.x8 8.2 MB/s | 3.6 MB 00:00 2023-06-28T20:04:47.675 INFO:teuthology.orchestra.run.smithi043.stdout:(131/131): librados2-18.0.0-4628.ga7bcf4ac.el8. 6.6 MB/s | 3.7 MB 00:00 2023-06-28T20:04:47.678 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2023-06-28T20:04:47.678 INFO:teuthology.orchestra.run.smithi043.stdout:Total 16 MB/s | 205 MB 00:13 2023-06-28T20:04:48.324 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2023-06-28T20:04:48.328 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2023-06-28T20:04:48.546 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2023-06-28T20:04:48.679 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2023-06-28T20:04:48.679 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2023-06-28T20:04:49.860 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2023-06-28T20:04:49.864 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2023-06-28T20:04:53.474 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2023-06-28T20:04:53.662 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-06-28T20:04:53.861 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-06-28T20:04:54.064 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-06-28T20:04:54.193 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-06-28T20:04:54.389 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-06-28T20:04:54.794 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-06-28T20:04:54.861 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2023-06-28T20:04:54.989 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librados2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 5/134 2023-06-28T20:04:55.057 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-06-28T20:04:55.251 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-06-28T20:04:55.394 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 5/134 2023-06-28T20:04:55.434 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-06-28T20:04:55.556 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-06-28T20:04:55.597 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librbd1-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 6/134 2023-06-28T20:04:55.706 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 6/134 2023-06-28T20:04:55.767 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-06-28T20:04:55.947 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 7/134 2023-06-28T20:04:56.045 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephfs2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 7/134 2023-06-28T20:04:56.194 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-06-28T20:04:56.235 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rados-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 8/134 2023-06-28T20:04:56.392 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librados2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 5/134 2023-06-28T20:04:56.467 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-06-28T20:04:56.692 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-06-28T20:04:56.824 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 5/134 2023-06-28T20:04:56.916 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-06-28T20:04:57.033 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librbd1-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 6/134 2023-06-28T20:04:57.172 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 6/134 2023-06-28T20:04:57.295 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-06-28T20:04:57.350 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 7/134 2023-06-28T20:04:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 7/134 2023-06-28T20:04:57.573 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-06-28T20:04:57.589 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rados-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 8/134 2023-06-28T20:04:57.752 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-06-28T20:04:57.755 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephsqlite-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 14/134 2023-06-28T20:04:57.860 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 14/134 2023-06-28T20:04:57.968 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-06-28T20:04:58.071 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libradosstriper1-2:18.0.0-4628.ga7bcf4ac.el8.x86 15/134 2023-06-28T20:04:58.187 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-06-28T20:04:58.220 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4628.ga7bcf4ac.el8.x86 15/134 2023-06-28T20:04:58.408 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-06-28T20:04:58.577 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-06-28T20:04:58.591 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-06-28T20:04:58.764 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-06-28T20:04:58.854 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-06-28T20:04:58.871 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-06-28T20:04:59.017 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephsqlite-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 14/134 2023-06-28T20:04:59.024 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-06-28T20:04:59.116 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 14/134 2023-06-28T20:04:59.138 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-06-28T20:04:59.275 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libradosstriper1-2:18.0.0-4628.ga7bcf4ac.el8.x86 15/134 2023-06-28T20:04:59.433 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4628.ga7bcf4ac.el8.x86 15/134 2023-06-28T20:04:59.613 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-06-28T20:04:59.776 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-06-28T20:04:59.790 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-06-28T20:04:59.946 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-06-28T20:04:59.967 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-06-28T20:05:00.325 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-06-28T20:05:00.482 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-06-28T20:05:00.552 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-06-28T20:05:00.616 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-06-28T20:05:00.796 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-06-28T20:05:01.279 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-06-28T20:05:01.481 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-06-28T20:05:01.482 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-06-28T20:05:01.737 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-06-28T20:05:01.910 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-06-28T20:05:02.101 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-06-28T20:05:02.172 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-06-28T20:05:02.282 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-06-28T20:05:02.352 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-common-2:18.0.0-4628.ga7bcf4ac.el8. 26/134 2023-06-28T20:05:02.493 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-argparse-2:18.0.0-4628.ga7bcf4ac.el 27/134 2023-06-28T20:05:02.766 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cephfs-2:18.0.0-4628.ga7bcf4ac.el8.x86_6 28/134 2023-06-28T20:05:02.971 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-06-28T20:05:03.156 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-06-28T20:05:03.427 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-06-28T20:05:03.623 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-06-28T20:05:03.840 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-06-28T20:05:04.006 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-common-2:18.0.0-4628.ga7bcf4ac.el8. 26/134 2023-06-28T20:05:04.146 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-argparse-2:18.0.0-4628.ga7bcf4ac.el 27/134 2023-06-28T20:05:04.391 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cephfs-2:18.0.0-4628.ga7bcf4ac.el8.x86_6 28/134 2023-06-28T20:05:04.607 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-06-28T20:05:04.726 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-06-28T20:05:05.763 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-06-28T20:05:06.060 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-06-28T20:05:06.355 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-06-28T20:05:06.505 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-06-28T20:05:06.557 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-06-28T20:05:06.806 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-06-28T20:05:06.996 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-06-28T20:05:07.192 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rbd-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 36/134 2023-06-28T20:05:07.368 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librados-devel-2:18.0.0-4628.ga7bcf4ac.el8.x86_6 37/134 2023-06-28T20:05:07.518 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-06-28T20:05:07.549 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-06-28T20:05:07.646 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-06-28T20:05:07.834 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-06-28T20:05:07.871 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-06-28T20:05:07.910 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-06-28T20:05:08.115 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-06-28T20:05:08.170 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-06-28T20:05:08.322 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-06-28T20:05:08.394 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-06-28T20:05:08.511 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-06-28T20:05:08.588 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-06-28T20:05:08.747 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-06-28T20:05:08.832 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-06-28T20:05:08.936 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-06-28T20:05:09.031 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-06-28T20:05:09.097 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rbd-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 36/134 2023-06-28T20:05:09.198 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-06-28T20:05:09.240 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librados-devel-2:18.0.0-4628.ga7bcf4ac.el8.x86_6 37/134 2023-06-28T20:05:09.374 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-06-28T20:05:09.400 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-06-28T20:05:09.518 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-06-28T20:05:09.614 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-06-28T20:05:09.706 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-06-28T20:05:09.756 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-06-28T20:05:09.783 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-06-28T20:05:09.960 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-06-28T20:05:10.008 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-06-28T20:05:10.134 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-06-28T20:05:10.228 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-06-28T20:05:10.373 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-06-28T20:05:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-06-28T20:05:10.471 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-06-28T20:05:10.658 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-06-28T20:05:10.811 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-06-28T20:05:10.837 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-06-28T20:05:10.983 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-06-28T20:05:11.055 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-06-28T20:05:11.142 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-06-28T20:05:11.309 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-06-28T20:05:11.445 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-06-28T20:05:11.616 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-06-28T20:05:11.671 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-06-28T20:05:11.833 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-06-28T20:05:11.910 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-06-28T20:05:12.212 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-06-28T20:05:12.234 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-06-28T20:05:12.435 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-06-28T20:05:12.518 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-06-28T20:05:12.647 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-06-28T20:05:12.744 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-06-28T20:05:12.869 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-06-28T20:05:12.978 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-06-28T20:05:13.033 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-06-28T20:05:13.190 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-06-28T20:05:13.206 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-06-28T20:05:13.486 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-06-28T20:05:13.512 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-06-28T20:05:13.706 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-06-28T20:05:13.800 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-06-28T20:05:13.908 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-06-28T20:05:14.031 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-06-28T20:05:14.108 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-06-28T20:05:14.261 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-06-28T20:05:14.298 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-06-28T20:05:14.401 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-06-28T20:05:14.470 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-06-28T20:05:14.571 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-06-28T20:05:14.781 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-06-28T20:05:14.881 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-06-28T20:05:15.045 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-06-28T20:05:15.100 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-06-28T20:05:15.256 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-06-28T20:05:15.272 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-06-28T20:05:15.453 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-06-28T20:05:15.491 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-06-28T20:05:15.639 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-06-28T20:05:15.642 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-06-28T20:05:15.819 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-06-28T20:05:15.820 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-06-28T20:05:16.019 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-06-28T20:05:16.090 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-06-28T20:05:16.179 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-06-28T20:05:16.313 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-06-28T20:05:16.324 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-06-28T20:05:16.452 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-06-28T20:05:16.551 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-06-28T20:05:16.593 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-06-28T20:05:16.769 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-06-28T20:05:16.770 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-06-28T20:05:16.931 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-06-28T20:05:16.963 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-06-28T20:05:17.145 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-06-28T20:05:17.216 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-06-28T20:05:17.292 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-06-28T20:05:17.433 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-06-28T20:05:17.445 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-06-28T20:05:17.607 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-06-28T20:05:17.657 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-06-28T20:05:17.838 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-06-28T20:05:17.878 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-06-28T20:05:18.053 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-06-28T20:05:18.310 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-06-28T20:05:18.539 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-06-28T20:05:18.723 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-06-28T20:05:18.895 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-06-28T20:05:18.929 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-06-28T20:05:19.146 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-06-28T20:05:19.375 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-06-28T20:05:19.718 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-06-28T20:05:20.026 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-06-28T20:05:20.069 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-06-28T20:05:20.224 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-06-28T20:05:20.320 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-06-28T20:05:20.549 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-06-28T20:05:20.842 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-06-28T20:05:21.041 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-06-28T20:05:21.086 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-06-28T20:05:21.242 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librgw2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 92/134 2023-06-28T20:05:21.309 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-06-28T20:05:21.319 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librgw2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 92/134 2023-06-28T20:05:21.492 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rgw-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 93/134 2023-06-28T20:05:22.195 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-06-28T20:05:22.363 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librgw2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 92/134 2023-06-28T20:05:22.435 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librgw2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 92/134 2023-06-28T20:05:22.630 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rgw-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 93/134 2023-06-28T20:05:25.097 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 94/134 2023-06-28T20:05:25.342 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-common-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 94/134 2023-06-28T20:05:25.937 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 94/134 2023-06-28T20:05:26.211 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-base-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 95/134 2023-06-28T20:05:26.295 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-base-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 95/134 2023-06-28T20:05:26.483 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-selinux-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 96/134 2023-06-28T20:05:26.498 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 94/134 2023-06-28T20:05:27.121 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-common-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 94/134 2023-06-28T20:05:27.744 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 94/134 2023-06-28T20:05:28.013 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-base-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 95/134 2023-06-28T20:05:28.094 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-base-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 95/134 2023-06-28T20:05:28.220 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-selinux-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 96/134 2023-06-28T20:05:45.486 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 96/134 2023-06-28T20:05:45.486 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /sys 2023-06-28T20:05:45.486 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /proc 2023-06-28T20:05:45.487 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /dev 2023-06-28T20:05:45.487 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /run 2023-06-28T20:05:45.487 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /mnt 2023-06-28T20:05:45.487 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /var/tmp 2023-06-28T20:05:45.487 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /home 2023-06-28T20:05:45.487 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /tmp 2023-06-28T20:05:45.487 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /dev 2023-06-28T20:05:45.488 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:05:45.830 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-volume-2:18.0.0-4628.ga7bcf4ac.el8.noarch 97/134 2023-06-28T20:05:47.414 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-volume-2:18.0.0-4628.ga7bcf4ac.el8.noarch 97/134 2023-06-28T20:05:47.623 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-osd-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 98/134 2023-06-28T20:05:47.699 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 96/134 2023-06-28T20:05:47.699 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /sys 2023-06-28T20:05:47.699 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /proc 2023-06-28T20:05:47.699 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /dev 2023-06-28T20:05:47.699 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /run 2023-06-28T20:05:47.700 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /mnt 2023-06-28T20:05:47.700 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /var/tmp 2023-06-28T20:05:47.700 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /home 2023-06-28T20:05:47.700 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /tmp 2023-06-28T20:05:47.700 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /dev 2023-06-28T20:05:47.700 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:05:47.841 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-osd-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 98/134 2023-06-28T20:05:47.979 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mds-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 99/134 2023-06-28T20:05:48.035 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-volume-2:18.0.0-4628.ga7bcf4ac.el8.noarch 97/134 2023-06-28T20:05:48.433 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mds-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 99/134 2023-06-28T20:05:48.637 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mon-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 100/134 2023-06-28T20:05:48.654 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mon-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 100/134 2023-06-28T20:05:49.724 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-volume-2:18.0.0-4628.ga7bcf4ac.el8.noarch 97/134 2023-06-28T20:05:49.910 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-osd-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 98/134 2023-06-28T20:05:50.141 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-osd-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 98/134 2023-06-28T20:05:50.282 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephadm-2:18.0.0-4628.ga7bcf4ac.el8.noarch 101/134 2023-06-28T20:05:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mds-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 99/134 2023-06-28T20:05:50.517 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephadm-2:18.0.0-4628.ga7bcf4ac.el8.noarch 101/134 2023-06-28T20:05:50.651 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4628.ga7bcf4ac.e 102/134 2023-06-28T20:05:50.767 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mds-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 99/134 2023-06-28T20:05:50.839 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4628.ga7bcf4ac. 103/134 2023-06-28T20:05:50.933 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mon-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 100/134 2023-06-28T20:05:50.951 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mon-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 100/134 2023-06-28T20:05:51.037 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4628.ga7bcf4ac.el8.noa 104/134 2023-06-28T20:05:51.251 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4628.ga7bcf4ac.el8.noa 104/134 2023-06-28T20:05:51.581 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4628.ga7bcf4ac.el8.n 105/134 2023-06-28T20:05:52.540 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4628.ga7bcf4ac.el8.n 105/134 2023-06-28T20:05:52.701 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:18.0.0-4628.ga7bcf4ac.el8.noarch 101/134 2023-06-28T20:05:52.728 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4628.ga7b 106/134 2023-06-28T20:05:52.818 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4628.ga7b 106/134 2023-06-28T20:05:52.887 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephadm-2:18.0.0-4628.ga7bcf4ac.el8.noarch 101/134 2023-06-28T20:05:53.032 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4628.ga7bcf4ac.e 102/134 2023-06-28T20:05:53.192 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4628.ga7bcf4ac.el 107/134 2023-06-28T20:05:53.226 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4628.ga7bcf4ac. 103/134 2023-06-28T20:05:53.391 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4628.ga7bcf4ac.el8.noa 104/134 2023-06-28T20:05:53.406 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 108/134 2023-06-28T20:05:53.441 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 108/134 2023-06-28T20:05:53.614 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4628.ga7bcf4ac.el8.noa 104/134 2023-06-28T20:05:53.636 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-rook-2:18.0.0-4628.ga7bcf4ac.el8.noarch 109/134 2023-06-28T20:05:53.647 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4628.ga7bcf4ac.el8.noarch 109/134 2023-06-28T20:05:53.934 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4628.ga7bcf4ac.el8.n 105/134 2023-06-28T20:05:54.953 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 110/134 2023-06-28T20:05:54.957 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4628.ga7bcf4ac.el8.n 105/134 2023-06-28T20:05:55.116 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4628.ga7b 106/134 2023-06-28T20:05:55.173 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-radosgw-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 111/134 2023-06-28T20:05:55.207 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 111/134 2023-06-28T20:05:55.211 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4628.ga7b 106/134 2023-06-28T20:05:55.361 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4628.ga7bcf 112/134 2023-06-28T20:05:55.408 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4628.ga7bcf 112/134 2023-06-28T20:05:55.546 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4628.ga7bcf4ac.el 107/134 2023-06-28T20:05:55.595 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-mirror-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 113/134 2023-06-28T20:05:55.701 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 108/134 2023-06-28T20:05:55.737 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 108/134 2023-06-28T20:05:55.899 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-rook-2:18.0.0-4628.ga7bcf4ac.el8.noarch 109/134 2023-06-28T20:05:55.911 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4628.ga7bcf4ac.el8.noarch 109/134 2023-06-28T20:05:55.938 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 113/134 2023-06-28T20:05:56.144 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-mirror-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 114/134 2023-06-28T20:05:57.299 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 110/134 2023-06-28T20:05:57.499 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-radosgw-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 111/134 2023-06-28T20:05:57.533 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 111/134 2023-06-28T20:05:57.691 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4628.ga7bcf 112/134 2023-06-28T20:05:57.736 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4628.ga7bcf 112/134 2023-06-28T20:05:57.858 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-mirror-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 113/134 2023-06-28T20:05:58.224 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 113/134 2023-06-28T20:05:58.374 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-mirror-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 114/134 2023-06-28T20:06:00.692 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 114/134 2023-06-28T20:06:01.579 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-test-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 115/134 2023-06-28T20:06:01.906 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-06-28T20:06:02.106 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-fuse-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 117/134 2023-06-28T20:06:02.358 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-06-28T20:06:02.551 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-06-28T20:06:02.744 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-06-28T20:06:02.972 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-06-28T20:06:03.172 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs-devel-2:18.0.0-4628.ga7bcf4ac.el8.x86_ 121/134 2023-06-28T20:06:03.213 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 114/134 2023-06-28T20:06:03.335 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-06-28T20:06:04.124 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-test-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 115/134 2023-06-28T20:06:04.443 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-06-28T20:06:04.619 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-fuse-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 117/134 2023-06-28T20:06:04.890 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-06-28T20:06:05.039 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-06-28T20:06:05.242 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-06-28T20:06:05.480 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-28T20:06:05.486 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-06-28T20:06:05.636 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs-devel-2:18.0.0-4628.ga7bcf4ac.el8.x86_ 121/134 2023-06-28T20:06:05.734 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-28T20:06:05.752 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-28T20:06:05.782 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-06-28T20:06:05.960 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephfs-top-2:18.0.0-4628.ga7bcf4ac.el8.noarch 124/134 2023-06-28T20:06:06.200 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-fuse-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 125/134 2023-06-28T20:06:06.481 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-nbd-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 126/134 2023-06-28T20:06:06.811 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-06-28T20:06:06.967 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-06-28T20:06:07.053 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-06-28T20:06:07.242 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-06-28T20:06:07.255 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-06-28T20:06:07.409 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-06-28T20:06:07.417 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-06-28T20:06:07.544 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-06-28T20:06:07.544 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-28T20:06:07.753 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-28T20:06:07.754 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-28T20:06:07.972 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-28T20:06:08.140 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-28T20:06:08.160 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-28T20:06:08.406 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-28T20:06:08.423 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-28T20:06:08.656 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-top-2:18.0.0-4628.ga7bcf4ac.el8.noarch 124/134 2023-06-28T20:06:08.665 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-06-28T20:06:08.666 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 1/134 2023-06-28T20:06:08.666 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-base-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2/134 2023-06-28T20:06:08.666 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-common-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 3/134 2023-06-28T20:06:08.666 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-fuse-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 4/134 2023-06-28T20:06:08.666 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4628.ga7bcf 5/134 2023-06-28T20:06:08.666 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mds-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 6/134 2023-06-28T20:06:08.667 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 7/134 2023-06-28T20:06:08.667 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mon-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 8/134 2023-06-28T20:06:08.667 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-osd-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 9/134 2023-06-28T20:06:08.667 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-radosgw-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 10/134 2023-06-28T20:06:08.667 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-selinux-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 11/134 2023-06-28T20:06:08.667 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-test-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 12/134 2023-06-28T20:06:08.667 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-mirror-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 13/134 2023-06-28T20:06:08.668 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs-devel-2:18.0.0-4628.ga7bcf4ac.el8.x86_ 14/134 2023-06-28T20:06:08.668 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 15/134 2023-06-28T20:06:08.668 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephsqlite-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 16/134 2023-06-28T20:06:08.668 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados-devel-2:18.0.0-4628.ga7bcf4ac.el8.x86_6 17/134 2023-06-28T20:06:08.668 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libradosstriper1-2:18.0.0-4628.ga7bcf4ac.el8.x86 18/134 2023-06-28T20:06:08.668 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librgw2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 19/134 2023-06-28T20:06:08.668 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4628.ga7bcf4ac.el 20/134 2023-06-28T20:06:08.669 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-common-2:18.0.0-4628.ga7bcf4ac.el8. 21/134 2023-06-28T20:06:08.669 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cephfs-2:18.0.0-4628.ga7bcf4ac.el8.x86_6 22/134 2023-06-28T20:06:08.669 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rados-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 23/134 2023-06-28T20:06:08.669 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rbd-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 24/134 2023-06-28T20:06:08.669 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rgw-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 25/134 2023-06-28T20:06:08.669 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-fuse-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 26/134 2023-06-28T20:06:08.669 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-mirror-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 27/134 2023-06-28T20:06:08.669 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-nbd-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 28/134 2023-06-28T20:06:08.670 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4628.ga7bcf4ac. 29/134 2023-06-28T20:06:08.670 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4628.ga7bcf4ac.el8.noa 30/134 2023-06-28T20:06:08.677 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4628.ga7bcf4ac.el8.n 31/134 2023-06-28T20:06:08.677 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4628.ga7b 32/134 2023-06-28T20:06:08.677 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4628.ga7bcf4ac.el 33/134 2023-06-28T20:06:08.677 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4628.ga7bcf4ac.el8.noarch 34/134 2023-06-28T20:06:08.677 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4628.ga7bcf4ac.e 35/134 2023-06-28T20:06:08.677 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-volume-2:18.0.0-4628.ga7bcf4ac.el8.noarch 36/134 2023-06-28T20:06:08.678 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephadm-2:18.0.0-4628.ga7bcf4ac.el8.noarch 37/134 2023-06-28T20:06:08.678 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephfs-top-2:18.0.0-4628.ga7bcf4ac.el8.noarch 38/134 2023-06-28T20:06:08.678 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-06-28T20:06:08.678 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-06-28T20:06:08.678 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-06-28T20:06:08.678 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-06-28T20:06:08.678 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-06-28T20:06:08.678 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-06-28T20:06:08.679 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-06-28T20:06:08.679 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-06-28T20:06:08.679 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-06-28T20:06:08.679 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-06-28T20:06:08.679 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-06-28T20:06:08.679 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-06-28T20:06:08.679 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-06-28T20:06:08.679 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-06-28T20:06:08.680 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-06-28T20:06:08.680 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-06-28T20:06:08.680 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-06-28T20:06:08.680 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-06-28T20:06:08.680 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-06-28T20:06:08.680 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-06-28T20:06:08.680 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-06-28T20:06:08.681 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-06-28T20:06:08.681 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-06-28T20:06:08.681 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-06-28T20:06:08.681 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-06-28T20:06:08.681 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-06-28T20:06:08.681 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-06-28T20:06:08.681 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-06-28T20:06:08.682 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-06-28T20:06:08.682 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-06-28T20:06:08.682 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-06-28T20:06:08.682 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-06-28T20:06:08.682 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-06-28T20:06:08.682 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-06-28T20:06:08.682 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-06-28T20:06:08.682 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-06-28T20:06:08.683 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-06-28T20:06:08.683 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-06-28T20:06:08.683 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-06-28T20:06:08.683 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-06-28T20:06:08.683 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-06-28T20:06:08.683 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-06-28T20:06:08.683 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-06-28T20:06:08.683 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-06-28T20:06:08.684 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-06-28T20:06:08.684 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-06-28T20:06:08.684 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-06-28T20:06:08.684 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-06-28T20:06:08.684 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-06-28T20:06:08.684 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-06-28T20:06:08.684 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-06-28T20:06:08.685 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-06-28T20:06:08.685 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-06-28T20:06:08.685 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-06-28T20:06:08.685 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-06-28T20:06:08.685 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-06-28T20:06:08.685 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-06-28T20:06:08.685 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-06-28T20:06:08.685 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-06-28T20:06:08.686 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-06-28T20:06:08.686 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-06-28T20:06:08.686 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-06-28T20:06:08.686 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-06-28T20:06:08.686 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-06-28T20:06:08.686 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-06-28T20:06:08.686 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-06-28T20:06:08.686 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-06-28T20:06:08.687 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-06-28T20:06:08.687 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-06-28T20:06:08.687 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-06-28T20:06:08.687 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-06-28T20:06:08.687 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-06-28T20:06:08.687 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-06-28T20:06:08.687 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-06-28T20:06:08.687 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-06-28T20:06:08.688 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-06-28T20:06:08.688 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-06-28T20:06:08.688 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-06-28T20:06:08.688 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-06-28T20:06:08.688 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-06-28T20:06:08.688 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-06-28T20:06:08.688 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-06-28T20:06:08.689 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-06-28T20:06:08.689 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-06-28T20:06:08.689 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-06-28T20:06:08.689 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-06-28T20:06:08.689 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-06-28T20:06:08.689 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-06-28T20:06:08.689 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-06-28T20:06:08.689 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-06-28T20:06:08.690 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 129/134 2023-06-28T20:06:08.690 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-06-28T20:06:08.690 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 131/134 2023-06-28T20:06:08.690 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-28T20:06:08.690 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-06-28T20:06:08.831 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-fuse-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 125/134 2023-06-28T20:06:09.110 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-nbd-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 126/134 2023-06-28T20:06:09.457 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-06-28T20:06:09.647 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-06-28T20:06:09.735 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-06-28T20:06:09.922 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-06-28T20:06:09.935 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-06-28T20:06:10.072 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-06-28T20:06:10.080 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-06-28T20:06:10.182 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-06-28T20:06:10.183 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-28T20:06:10.409 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-28T20:06:10.409 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-28T20:06:10.654 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-28T20:06:10.830 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-28T20:06:11.224 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-28T20:06:11.224 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:51 ago on Wed 28 Jun 2023 08:04:20 PM UTC. 2023-06-28T20:06:11.385 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-06-28T20:06:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 1/134 2023-06-28T20:06:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-base-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2/134 2023-06-28T20:06:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-common-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 3/134 2023-06-28T20:06:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-fuse-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 4/134 2023-06-28T20:06:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4628.ga7bcf 5/134 2023-06-28T20:06:11.386 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mds-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 6/134 2023-06-28T20:06:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 7/134 2023-06-28T20:06:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mon-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 8/134 2023-06-28T20:06:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-osd-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 9/134 2023-06-28T20:06:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-radosgw-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 10/134 2023-06-28T20:06:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-selinux-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 11/134 2023-06-28T20:06:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-test-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 12/134 2023-06-28T20:06:11.387 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-mirror-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 13/134 2023-06-28T20:06:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs-devel-2:18.0.0-4628.ga7bcf4ac.el8.x86_ 14/134 2023-06-28T20:06:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 15/134 2023-06-28T20:06:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephsqlite-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 16/134 2023-06-28T20:06:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados-devel-2:18.0.0-4628.ga7bcf4ac.el8.x86_6 17/134 2023-06-28T20:06:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libradosstriper1-2:18.0.0-4628.ga7bcf4ac.el8.x86 18/134 2023-06-28T20:06:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librgw2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 19/134 2023-06-28T20:06:11.388 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4628.ga7bcf4ac.el 20/134 2023-06-28T20:06:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-common-2:18.0.0-4628.ga7bcf4ac.el8. 21/134 2023-06-28T20:06:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cephfs-2:18.0.0-4628.ga7bcf4ac.el8.x86_6 22/134 2023-06-28T20:06:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rados-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 23/134 2023-06-28T20:06:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rbd-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 24/134 2023-06-28T20:06:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rgw-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 25/134 2023-06-28T20:06:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-fuse-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 26/134 2023-06-28T20:06:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-mirror-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 27/134 2023-06-28T20:06:11.389 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-nbd-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 28/134 2023-06-28T20:06:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4628.ga7bcf4ac. 29/134 2023-06-28T20:06:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4628.ga7bcf4ac.el8.noa 30/134 2023-06-28T20:06:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4628.ga7bcf4ac.el8.n 31/134 2023-06-28T20:06:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4628.ga7b 32/134 2023-06-28T20:06:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4628.ga7bcf4ac.el 33/134 2023-06-28T20:06:11.390 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4628.ga7bcf4ac.el8.noarch 34/134 2023-06-28T20:06:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4628.ga7bcf4ac.e 35/134 2023-06-28T20:06:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-volume-2:18.0.0-4628.ga7bcf4ac.el8.noarch 36/134 2023-06-28T20:06:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephadm-2:18.0.0-4628.ga7bcf4ac.el8.noarch 37/134 2023-06-28T20:06:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-top-2:18.0.0-4628.ga7bcf4ac.el8.noarch 38/134 2023-06-28T20:06:11.391 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-06-28T20:06:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-06-28T20:06:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-06-28T20:06:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-06-28T20:06:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-06-28T20:06:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-06-28T20:06:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-06-28T20:06:11.392 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-06-28T20:06:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-06-28T20:06:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-06-28T20:06:11.393 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-06-28T20:06:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-06-28T20:06:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-06-28T20:06:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-06-28T20:06:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-06-28T20:06:11.394 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-06-28T20:06:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-06-28T20:06:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-06-28T20:06:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-06-28T20:06:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-06-28T20:06:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-06-28T20:06:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-06-28T20:06:11.395 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-06-28T20:06:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-06-28T20:06:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-06-28T20:06:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-06-28T20:06:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-06-28T20:06:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-06-28T20:06:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-06-28T20:06:11.396 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-06-28T20:06:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-06-28T20:06:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-06-28T20:06:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-06-28T20:06:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-06-28T20:06:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-06-28T20:06:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-06-28T20:06:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-06-28T20:06:11.397 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-06-28T20:06:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-06-28T20:06:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-06-28T20:06:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-06-28T20:06:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-06-28T20:06:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-06-28T20:06:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-06-28T20:06:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-06-28T20:06:11.398 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-06-28T20:06:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-06-28T20:06:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-06-28T20:06:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-06-28T20:06:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-06-28T20:06:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-06-28T20:06:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-06-28T20:06:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-06-28T20:06:11.399 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-06-28T20:06:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-06-28T20:06:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-06-28T20:06:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-06-28T20:06:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-06-28T20:06:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-06-28T20:06:11.400 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-06-28T20:06:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-06-28T20:06:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-06-28T20:06:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-06-28T20:06:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-06-28T20:06:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-06-28T20:06:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-06-28T20:06:11.401 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-06-28T20:06:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-06-28T20:06:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-06-28T20:06:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-06-28T20:06:11.402 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-06-28T20:06:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-06-28T20:06:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-06-28T20:06:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-06-28T20:06:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-06-28T20:06:11.403 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-06-28T20:06:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-06-28T20:06:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-06-28T20:06:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-06-28T20:06:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-06-28T20:06:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-06-28T20:06:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-06-28T20:06:11.404 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-06-28T20:06:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-06-28T20:06:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-06-28T20:06:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-06-28T20:06:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-06-28T20:06:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-06-28T20:06:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-06-28T20:06:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-06-28T20:06:11.405 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 129/134 2023-06-28T20:06:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-06-28T20:06:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 131/134 2023-06-28T20:06:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-28T20:06:11.406 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-06-28T20:06:13.817 INFO:teuthology.orchestra.run.smithi178.stdout:Installed products updated. 2023-06-28T20:06:13.907 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-28T20:06:13.917 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:51 ago on Wed 28 Jun 2023 08:04:22 PM UTC. 2023-06-28T20:06:14.053 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:14.054 INFO:teuthology.orchestra.run.smithi178.stdout:Upgraded: 2023-06-28T20:06:14.054 INFO:teuthology.orchestra.run.smithi178.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-06-28T20:06:14.054 INFO:teuthology.orchestra.run.smithi178.stdout: librados2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.054 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.054 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2023-06-28T20:06:14.054 INFO:teuthology.orchestra.run.smithi178.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-06-28T20:06:14.054 INFO:teuthology.orchestra.run.smithi178.stdout: automake-1.16.1-7.el8.noarch 2023-06-28T20:06:14.055 INFO:teuthology.orchestra.run.smithi178.stdout: bison-3.0.4-10.el8.x86_64 2023-06-28T20:06:14.055 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.055 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.055 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.055 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.055 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards-2:18.0.0-4628.ga7bcf4ac.el8.noarch 2023-06-28T20:06:14.055 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.056 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.056 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.056 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm-2:18.0.0-4628.ga7bcf4ac.el8.noarch 2023-06-28T20:06:14.056 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard-2:18.0.0-4628.ga7bcf4ac.el8.noarch 2023-06-28T20:06:14.056 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4628.ga7bcf4ac.el8.noarch 2023-06-28T20:06:14.056 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core-2:18.0.0-4628.ga7bcf4ac.el8.noarch 2023-06-28T20:06:14.056 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook-2:18.0.0-4628.ga7bcf4ac.el8.noarch 2023-06-28T20:06:14.057 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.057 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.057 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts-2:18.0.0-4628.ga7bcf4ac.el8.noarch 2023-06-28T20:06:14.057 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.057 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.057 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.057 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-volume-2:18.0.0-4628.ga7bcf4ac.el8.noarch 2023-06-28T20:06:14.057 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm-2:18.0.0-4628.ga7bcf4ac.el8.noarch 2023-06-28T20:06:14.058 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-mirror-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.058 INFO:teuthology.orchestra.run.smithi178.stdout: cephfs-top-2:18.0.0-4628.ga7bcf4ac.el8.noarch 2023-06-28T20:06:14.058 INFO:teuthology.orchestra.run.smithi178.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-06-28T20:06:14.058 INFO:teuthology.orchestra.run.smithi178.stdout: flex-2.6.1-9.el8.x86_64 2023-06-28T20:06:14.058 INFO:teuthology.orchestra.run.smithi178.stdout: fuse-2.9.7-15.el8.x86_64 2023-06-28T20:06:14.059 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-28T20:06:14.059 INFO:teuthology.orchestra.run.smithi178.stdout: jq-1.6-3.el8.x86_64 2023-06-28T20:06:14.059 INFO:teuthology.orchestra.run.smithi178.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-06-28T20:06:14.059 INFO:teuthology.orchestra.run.smithi178.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-06-28T20:06:14.059 INFO:teuthology.orchestra.run.smithi178.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-28T20:06:14.059 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.059 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.059 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.060 INFO:teuthology.orchestra.run.smithi178.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-06-28T20:06:14.060 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-06-28T20:06:14.060 INFO:teuthology.orchestra.run.smithi178.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-28T20:06:14.060 INFO:teuthology.orchestra.run.smithi178.stdout: libpq-13.5-1.el8.x86_64 2023-06-28T20:06:14.061 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-06-28T20:06:14.061 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-28T20:06:14.061 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.061 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.061 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-06-28T20:06:14.061 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.061 INFO:teuthology.orchestra.run.smithi178.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-28T20:06:14.062 INFO:teuthology.orchestra.run.smithi178.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-06-28T20:06:14.062 INFO:teuthology.orchestra.run.smithi178.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-28T20:06:14.062 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-28T20:06:14.062 INFO:teuthology.orchestra.run.smithi178.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-06-28T20:06:14.062 INFO:teuthology.orchestra.run.smithi178.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-28T20:06:14.062 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-28T20:06:14.062 INFO:teuthology.orchestra.run.smithi178.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-28T20:06:14.063 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-0.3.15-3.el8.x86_64 2023-06-28T20:06:14.063 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-06-28T20:06:14.063 INFO:teuthology.orchestra.run.smithi178.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-06-28T20:06:14.063 INFO:teuthology.orchestra.run.smithi178.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-28T20:06:14.063 INFO:teuthology.orchestra.run.smithi178.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-28T20:06:14.063 INFO:teuthology.orchestra.run.smithi178.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-28T20:06:14.063 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-28T20:06:14.064 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-28T20:06:14.064 INFO:teuthology.orchestra.run.smithi178.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-28T20:06:14.064 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-28T20:06:14.064 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel-2.5.1-7.el8.noarch 2023-06-28T20:06:14.064 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-28T20:06:14.064 INFO:teuthology.orchestra.run.smithi178.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-28T20:06:14.064 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-28T20:06:14.065 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.065 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.065 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.065 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-28T20:06:14.065 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-06-28T20:06:14.065 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-28T20:06:14.065 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-28T20:06:14.066 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-06-28T20:06:14.066 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-28T20:06:14.066 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-28T20:06:14.066 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-28T20:06:14.066 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-06-28T20:06:14.066 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-28T20:06:14.066 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-28T20:06:14.067 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-28T20:06:14.067 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-28T20:06:14.067 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-28T20:06:14.067 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-28T20:06:14.067 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-28T20:06:14.067 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-28T20:06:14.067 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-28T20:06:14.067 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-06-28T20:06:14.068 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-06-28T20:06:14.068 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-28T20:06:14.068 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-28T20:06:14.068 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend-2.6-1.el8.noarch 2023-06-28T20:06:14.068 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-28T20:06:14.068 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-06-28T20:06:14.069 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-28T20:06:14.069 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-28T20:06:14.069 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser-2.14-14.el8.noarch 2023-06-28T20:06:14.069 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-28T20:06:14.069 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.069 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.069 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-28T20:06:14.070 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-28T20:06:14.070 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.070 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-28T20:06:14.070 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-28T20:06:14.070 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-06-28T20:06:14.070 INFO:teuthology.orchestra.run.smithi178.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-28T20:06:14.070 INFO:teuthology.orchestra.run.smithi178.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-28T20:06:14.071 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-28T20:06:14.071 INFO:teuthology.orchestra.run.smithi178.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-28T20:06:14.071 INFO:teuthology.orchestra.run.smithi178.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-28T20:06:14.071 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-28T20:06:14.071 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-28T20:06:14.071 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-28T20:06:14.071 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-28T20:06:14.071 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-28T20:06:14.072 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.072 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.072 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:14.072 INFO:teuthology.orchestra.run.smithi178.stdout: socat-1.7.4.1-1.el8.x86_64 2023-06-28T20:06:14.072 INFO:teuthology.orchestra.run.smithi178.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-28T20:06:14.072 INFO:teuthology.orchestra.run.smithi178.stdout: uuid-1.6.2-43.el8.x86_64 2023-06-28T20:06:14.072 INFO:teuthology.orchestra.run.smithi178.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-06-28T20:06:14.073 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-28T20:06:14.073 INFO:teuthology.orchestra.run.smithi178.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-06-28T20:06:14.073 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:14.073 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2023-06-28T20:06:14.279 DEBUG:teuthology.parallel:result is None 2023-06-28T20:06:16.668 INFO:teuthology.orchestra.run.smithi043.stdout:Installed products updated. 2023-06-28T20:06:16.910 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:16.911 INFO:teuthology.orchestra.run.smithi043.stdout:Upgraded: 2023-06-28T20:06:16.911 INFO:teuthology.orchestra.run.smithi043.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-06-28T20:06:16.911 INFO:teuthology.orchestra.run.smithi043.stdout: librados2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.911 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.911 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2023-06-28T20:06:16.911 INFO:teuthology.orchestra.run.smithi043.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-06-28T20:06:16.912 INFO:teuthology.orchestra.run.smithi043.stdout: automake-1.16.1-7.el8.noarch 2023-06-28T20:06:16.912 INFO:teuthology.orchestra.run.smithi043.stdout: bison-3.0.4-10.el8.x86_64 2023-06-28T20:06:16.912 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.912 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.912 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.912 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.913 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards-2:18.0.0-4628.ga7bcf4ac.el8.noarch 2023-06-28T20:06:16.913 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.913 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.913 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.913 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm-2:18.0.0-4628.ga7bcf4ac.el8.noarch 2023-06-28T20:06:16.913 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard-2:18.0.0-4628.ga7bcf4ac.el8.noarch 2023-06-28T20:06:16.913 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4628.ga7bcf4ac.el8.noarch 2023-06-28T20:06:16.913 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core-2:18.0.0-4628.ga7bcf4ac.el8.noarch 2023-06-28T20:06:16.914 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook-2:18.0.0-4628.ga7bcf4ac.el8.noarch 2023-06-28T20:06:16.914 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.914 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.914 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts-2:18.0.0-4628.ga7bcf4ac.el8.noarch 2023-06-28T20:06:16.914 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.914 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.914 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.915 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume-2:18.0.0-4628.ga7bcf4ac.el8.noarch 2023-06-28T20:06:16.915 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm-2:18.0.0-4628.ga7bcf4ac.el8.noarch 2023-06-28T20:06:16.915 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-mirror-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.915 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-top-2:18.0.0-4628.ga7bcf4ac.el8.noarch 2023-06-28T20:06:16.915 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-06-28T20:06:16.915 INFO:teuthology.orchestra.run.smithi043.stdout: flex-2.6.1-9.el8.x86_64 2023-06-28T20:06:16.915 INFO:teuthology.orchestra.run.smithi043.stdout: fuse-2.9.7-15.el8.x86_64 2023-06-28T20:06:16.916 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-28T20:06:16.916 INFO:teuthology.orchestra.run.smithi043.stdout: jq-1.6-3.el8.x86_64 2023-06-28T20:06:16.916 INFO:teuthology.orchestra.run.smithi043.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-06-28T20:06:16.916 INFO:teuthology.orchestra.run.smithi043.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-06-28T20:06:16.916 INFO:teuthology.orchestra.run.smithi043.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-28T20:06:16.916 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.916 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.917 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.917 INFO:teuthology.orchestra.run.smithi043.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-06-28T20:06:16.917 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-06-28T20:06:16.917 INFO:teuthology.orchestra.run.smithi043.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-28T20:06:16.917 INFO:teuthology.orchestra.run.smithi043.stdout: libpq-13.5-1.el8.x86_64 2023-06-28T20:06:16.917 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-06-28T20:06:16.917 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-28T20:06:16.917 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.918 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.918 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-06-28T20:06:16.918 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.918 INFO:teuthology.orchestra.run.smithi043.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-28T20:06:16.918 INFO:teuthology.orchestra.run.smithi043.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-06-28T20:06:16.918 INFO:teuthology.orchestra.run.smithi043.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-28T20:06:16.918 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-28T20:06:16.918 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-06-28T20:06:16.919 INFO:teuthology.orchestra.run.smithi043.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-28T20:06:16.919 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-28T20:06:16.919 INFO:teuthology.orchestra.run.smithi043.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-28T20:06:16.919 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-0.3.15-3.el8.x86_64 2023-06-28T20:06:16.919 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-06-28T20:06:16.919 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-06-28T20:06:16.919 INFO:teuthology.orchestra.run.smithi043.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-28T20:06:16.919 INFO:teuthology.orchestra.run.smithi043.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-28T20:06:16.920 INFO:teuthology.orchestra.run.smithi043.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-28T20:06:16.920 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-28T20:06:16.920 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-28T20:06:16.920 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-28T20:06:16.920 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-28T20:06:16.920 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel-2.5.1-7.el8.noarch 2023-06-28T20:06:16.920 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-28T20:06:16.921 INFO:teuthology.orchestra.run.smithi043.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-28T20:06:16.921 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-28T20:06:16.921 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.921 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.921 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.921 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-28T20:06:16.921 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-06-28T20:06:16.922 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-28T20:06:16.922 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-28T20:06:16.922 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-06-28T20:06:16.922 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-28T20:06:16.922 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-28T20:06:16.922 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-28T20:06:16.922 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-06-28T20:06:16.922 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-28T20:06:16.923 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-28T20:06:16.923 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-28T20:06:16.923 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-28T20:06:16.923 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-28T20:06:16.923 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-28T20:06:16.923 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-28T20:06:16.923 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-28T20:06:16.923 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-28T20:06:16.924 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-06-28T20:06:16.924 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-06-28T20:06:16.924 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-28T20:06:16.924 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-28T20:06:16.924 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend-2.6-1.el8.noarch 2023-06-28T20:06:16.924 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-28T20:06:16.924 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-06-28T20:06:16.925 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-28T20:06:16.925 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-28T20:06:16.925 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser-2.14-14.el8.noarch 2023-06-28T20:06:16.925 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-28T20:06:16.925 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.925 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.925 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-28T20:06:16.925 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-28T20:06:16.926 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.926 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-28T20:06:16.926 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-28T20:06:16.926 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-06-28T20:06:16.926 INFO:teuthology.orchestra.run.smithi043.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-28T20:06:16.926 INFO:teuthology.orchestra.run.smithi043.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-28T20:06:16.926 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-28T20:06:16.927 INFO:teuthology.orchestra.run.smithi043.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-28T20:06:16.927 INFO:teuthology.orchestra.run.smithi043.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-28T20:06:16.927 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-28T20:06:16.927 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-28T20:06:16.927 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-28T20:06:16.927 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-28T20:06:16.927 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-28T20:06:16.928 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.928 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.928 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd-2:18.0.0-4628.ga7bcf4ac.el8.x86_64 2023-06-28T20:06:16.928 INFO:teuthology.orchestra.run.smithi043.stdout: socat-1.7.4.1-1.el8.x86_64 2023-06-28T20:06:16.928 INFO:teuthology.orchestra.run.smithi043.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-28T20:06:16.928 INFO:teuthology.orchestra.run.smithi043.stdout: uuid-1.6.2-43.el8.x86_64 2023-06-28T20:06:16.928 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-06-28T20:06:16.929 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-28T20:06:16.929 INFO:teuthology.orchestra.run.smithi043.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-06-28T20:06:16.929 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:16.929 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2023-06-28T20:06:17.104 DEBUG:teuthology.parallel:result is None 2023-06-28T20:06:17.105 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a7bcf4acb88d3aafb511a5f395764c81b8c88a7c 2023-06-28T20:06:17.253 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-28T20:06:17.278 INFO:teuthology.orchestra.run.smithi043.stdout:18.0.0-4628.ga7bcf4ac.el8 2023-06-28T20:06:17.279 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4628.ga7bcf4ac.el8 2023-06-28T20:06:17.279 INFO:teuthology.task.install:The correct ceph version 18.0.0-4628.ga7bcf4ac is installed. 2023-06-28T20:06:17.280 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a7bcf4acb88d3aafb511a5f395764c81b8c88a7c 2023-06-28T20:06:17.586 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-28T20:06:17.608 INFO:teuthology.orchestra.run.smithi178.stdout:18.0.0-4628.ga7bcf4ac.el8 2023-06-28T20:06:17.609 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4628.ga7bcf4ac.el8 2023-06-28T20:06:17.609 INFO:teuthology.task.install:The correct ceph version 18.0.0-4628.ga7bcf4ac is installed. 2023-06-28T20:06:17.610 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-06-28T20:06:17.610 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:06:17.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-28T20:06:17.652 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:06:17.653 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-28T20:06:17.693 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-06-28T20:06:17.693 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:06:17.694 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/daemon-helper 2023-06-28T20:06:17.722 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-28T20:06:17.789 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:06:17.790 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/daemon-helper 2023-06-28T20:06:17.817 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-28T20:06:17.884 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-06-28T20:06:17.887 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:06:17.887 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-28T20:06:17.916 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-28T20:06:17.984 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:06:17.985 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-28T20:06:18.012 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-28T20:06:18.080 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-06-28T20:06:18.080 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:06:18.081 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/stdin-killer 2023-06-28T20:06:18.108 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-28T20:06:18.174 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:06:18.174 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/stdin-killer 2023-06-28T20:06:18.200 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-28T20:06:18.268 INFO:teuthology.run_tasks:Running task ceph... 2023-06-28T20:06:18.340 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-06-28T20:06:18.341 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 777 /var/log/ceph 2023-06-28T20:06:18.343 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 777 /var/log/ceph 2023-06-28T20:06:18.368 INFO:tasks.ceph:Disabling ceph logrotate... 2023-06-28T20:06:18.369 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/logrotate.d/ceph 2023-06-28T20:06:18.410 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/logrotate.d/ceph 2023-06-28T20:06:18.437 INFO:tasks.ceph:Creating extra log directories... 2023-06-28T20:06:18.437 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-06-28T20:06:18.479 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-06-28T20:06:18.512 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-06-28T20:06:18.512 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-06-28T20:06:18.513 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-06-28T20:06:18.513 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:06:18.513 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-06-28T20:06:18.562 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-06-28T20:06:23.740 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-06-28T20:06:23.768 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:06:23.769 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-06-28T20:06:23.784 DEBUG:teuthology.orchestra.run.smithi178:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-06-28T20:06:24.276 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-06-28T20:06:24.308 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-06-28T20:06:24.308 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-06-28T20:06:24.308 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2023-06-28_10:04:29-fs-wip-vshankar-testing-20230628.043830-testing-default-smithi/7319991', 'branch': 'wip-vshankar-testing-20230628.043830', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7319991', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'd535508c699c563d11e77fb7b284c600a069451a'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2023-06-28_10:04:29-fs-wip-vshankar-testing-20230628.043830-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-20230628.043830'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'a7bcf4acb88d3aafb511a5f395764c81b8c88a7c'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a7bcf4acb88d3aafb511a5f395764c81b8c88a7c'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-20230628.043830', 'sha1': 'a7bcf4acb88d3aafb511a5f395764c81b8c88a7c'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': 'a7bcf4acb88d3aafb511a5f395764c81b8c88a7c', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-20230628.043830', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_a7bcf4acb88d3aafb511a5f395764c81b8c88a7c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'a7bcf4acb88d3aafb511a5f395764c81b8c88a7c', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi043.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=', 'smithi178.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'd535508c699c563d11e77fb7b284c600a069451a'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a7bcf4acb88d3aafb511a5f395764c81b8c88a7c'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_quota']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'aa9345a996190b15ab9d46b5969b5f8280bfba99', 'timestamp': '2023-06-28_10:04:29', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23459'} 2023-06-28T20:06:24.309 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-06-28T20:06:24.312 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-06-28T20:06:24.358 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m0777 -- /var/run/ceph 2023-06-28T20:06:24.367 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m0777 -- /var/run/ceph 2023-06-28T20:06:24.426 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:06:24.427 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/scratch_devs of=/dev/stdout 2023-06-28T20:06:24.450 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-28T20:06:24.451 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_1 2023-06-28T20:06:24.507 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-28T20:06:24.508 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-28T20:06:24.508 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 6h/6d Inode: 379721 Links: 1 2023-06-28T20:06:24.508 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-28T20:06:24.508 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2023-06-28T20:06:24.508 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2023-06-28 20:06:11.059284985 +0000 2023-06-28T20:06:24.509 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2023-06-28 19:56:58.055364033 +0000 2023-06-28T20:06:24.509 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2023-06-28 19:56:58.055364033 +0000 2023-06-28T20:06:24.509 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: - 2023-06-28T20:06:24.510 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-28T20:06:24.574 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2023-06-28T20:06:24.574 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2023-06-28T20:06:24.575 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000103375 s, 5.0 MB/s 2023-06-28T20:06:24.575 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-28T20:06:24.634 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_2 2023-06-28T20:06:24.692 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-28T20:06:24.692 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-28T20:06:24.692 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 6h/6d Inode: 375208 Links: 1 2023-06-28T20:06:24.693 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-28T20:06:24.693 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2023-06-28T20:06:24.693 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2023-06-28 20:06:11.059284985 +0000 2023-06-28T20:06:24.693 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2023-06-28 19:56:58.568358024 +0000 2023-06-28T20:06:24.694 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2023-06-28 19:56:58.568358024 +0000 2023-06-28T20:06:24.694 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: - 2023-06-28T20:06:24.695 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-28T20:06:24.757 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2023-06-28T20:06:24.758 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2023-06-28T20:06:24.758 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000101357 s, 5.1 MB/s 2023-06-28T20:06:24.759 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-28T20:06:24.817 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_3 2023-06-28T20:06:24.874 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-28T20:06:24.875 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-28T20:06:24.875 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 6h/6d Inode: 374448 Links: 1 2023-06-28T20:06:24.875 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-28T20:06:24.875 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2023-06-28T20:06:24.875 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2023-06-28 20:06:11.059284985 +0000 2023-06-28T20:06:24.875 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2023-06-28 19:56:59.081352014 +0000 2023-06-28T20:06:24.876 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2023-06-28 19:56:59.081352014 +0000 2023-06-28T20:06:24.876 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: - 2023-06-28T20:06:24.876 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-28T20:06:24.940 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2023-06-28T20:06:24.940 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2023-06-28T20:06:24.941 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000104877 s, 4.9 MB/s 2023-06-28T20:06:24.941 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-28T20:06:24.999 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_4 2023-06-28T20:06:25.058 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-28T20:06:25.058 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-28T20:06:25.058 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 6h/6d Inode: 382297 Links: 1 2023-06-28T20:06:25.058 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-28T20:06:25.058 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2023-06-28T20:06:25.058 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2023-06-28 20:06:11.059284985 +0000 2023-06-28T20:06:25.059 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2023-06-28 19:56:59.598345958 +0000 2023-06-28T20:06:25.059 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2023-06-28 19:56:59.598345958 +0000 2023-06-28T20:06:25.059 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: - 2023-06-28T20:06:25.059 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-28T20:06:25.123 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2023-06-28T20:06:25.123 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2023-06-28T20:06:25.123 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000108014 s, 4.7 MB/s 2023-06-28T20:06:25.124 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-28T20:06:25.181 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-28T20:06:25.182 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:06:25.182 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/scratch_devs of=/dev/stdout 2023-06-28T20:06:25.198 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-28T20:06:25.198 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_1 2023-06-28T20:06:25.256 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-28T20:06:25.256 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-28T20:06:25.257 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 6h/6d Inode: 380566 Links: 1 2023-06-28T20:06:25.257 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-28T20:06:25.257 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2023-06-28T20:06:25.257 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2023-06-28 20:06:08.359538572 +0000 2023-06-28T20:06:25.257 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2023-06-28 20:03:12.657824221 +0000 2023-06-28T20:06:25.257 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2023-06-28 20:03:12.657824221 +0000 2023-06-28T20:06:25.257 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2023-06-28T20:06:25.258 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-28T20:06:25.321 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2023-06-28T20:06:25.321 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2023-06-28T20:06:25.321 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 9.7307e-05 s, 5.3 MB/s 2023-06-28T20:06:25.322 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-28T20:06:25.380 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_2 2023-06-28T20:06:25.437 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-28T20:06:25.438 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-28T20:06:25.438 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 6h/6d Inode: 378628 Links: 1 2023-06-28T20:06:25.438 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-28T20:06:25.438 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2023-06-28T20:06:25.438 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2023-06-28 20:06:08.359538572 +0000 2023-06-28T20:06:25.438 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2023-06-28 20:03:13.172814615 +0000 2023-06-28T20:06:25.439 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2023-06-28 20:03:13.172814615 +0000 2023-06-28T20:06:25.439 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2023-06-28T20:06:25.439 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-28T20:06:25.503 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2023-06-28T20:06:25.504 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2023-06-28T20:06:25.504 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000108506 s, 4.7 MB/s 2023-06-28T20:06:25.505 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-28T20:06:25.562 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_3 2023-06-28T20:06:25.619 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-28T20:06:25.619 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-28T20:06:25.619 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 6h/6d Inode: 378712 Links: 1 2023-06-28T20:06:25.619 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-28T20:06:25.619 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2023-06-28T20:06:25.619 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2023-06-28 20:06:08.359538572 +0000 2023-06-28T20:06:25.620 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2023-06-28 20:03:13.671805308 +0000 2023-06-28T20:06:25.620 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2023-06-28 20:03:13.671805308 +0000 2023-06-28T20:06:25.620 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2023-06-28T20:06:25.620 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-28T20:06:25.684 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2023-06-28T20:06:25.684 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2023-06-28T20:06:25.684 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000103774 s, 4.9 MB/s 2023-06-28T20:06:25.685 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-28T20:06:25.743 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_4 2023-06-28T20:06:25.800 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-28T20:06:25.801 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-28T20:06:25.801 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 6h/6d Inode: 382467 Links: 1 2023-06-28T20:06:25.801 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-28T20:06:25.801 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2023-06-28T20:06:25.801 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2023-06-28 20:06:08.359538572 +0000 2023-06-28T20:06:25.801 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2023-06-28 20:03:14.183795757 +0000 2023-06-28T20:06:25.802 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2023-06-28 20:03:14.183795757 +0000 2023-06-28T20:06:25.802 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2023-06-28T20:06:25.802 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-28T20:06:25.866 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2023-06-28T20:06:25.867 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2023-06-28T20:06:25.867 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000122213 s, 4.2 MB/s 2023-06-28T20:06:25.868 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-28T20:06:25.924 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-28T20:06:25.925 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi043.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi178.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-06-28T20:06:25.925 INFO:tasks.ceph:Generating config... 2023-06-28T20:06:25.926 INFO:tasks.ceph:[client] client mount timeout = 600 2023-06-28T20:06:25.927 INFO:tasks.ceph:[client] debug client = 20 2023-06-28T20:06:25.927 INFO:tasks.ceph:[client] debug ms = 1 2023-06-28T20:06:25.927 INFO:tasks.ceph:[client] pid file = 2023-06-28T20:06:25.927 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-06-28T20:06:25.927 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-06-28T20:06:25.928 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-06-28T20:06:25.929 INFO:tasks.ceph:[mds] debug mds = 20 2023-06-28T20:06:25.929 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-06-28T20:06:25.929 INFO:tasks.ceph:[mds] debug ms = 1 2023-06-28T20:06:25.929 INFO:tasks.ceph:[mds] mds debug frag = True 2023-06-28T20:06:25.929 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-06-28T20:06:25.930 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-06-28T20:06:25.930 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-06-28T20:06:25.930 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-06-28T20:06:25.930 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-06-28T20:06:25.930 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-06-28T20:06:25.930 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-06-28T20:06:25.931 INFO:tasks.ceph:[mgr] debug ms = 1 2023-06-28T20:06:25.931 INFO:tasks.ceph:[mon] debug mon = 20 2023-06-28T20:06:25.931 INFO:tasks.ceph:[mon] debug ms = 1 2023-06-28T20:06:25.931 INFO:tasks.ceph:[mon] debug paxos = 20 2023-06-28T20:06:25.931 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-06-28T20:06:25.932 INFO:tasks.ceph:[osd] bdev async discard = True 2023-06-28T20:06:25.932 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-06-28T20:06:25.932 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-06-28T20:06:25.932 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-06-28T20:06:25.932 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-06-28T20:06:25.933 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-06-28T20:06:25.933 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-06-28T20:06:25.933 INFO:tasks.ceph:[osd] debug ms = 1 2023-06-28T20:06:25.933 INFO:tasks.ceph:[osd] debug osd = 20 2023-06-28T20:06:25.933 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-06-28T20:06:25.934 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-06-28T20:06:25.934 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-06-28T20:06:25.934 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-06-28T20:06:25.935 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-06-28T20:06:25.935 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-06-28T20:06:25.935 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-06-28T20:06:25.935 INFO:tasks.ceph:Setting up mon.a... 2023-06-28T20:06:25.936 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-06-28T20:06:25.986 INFO:teuthology.orchestra.run.smithi043.stdout:creating /etc/ceph/ceph.keyring 2023-06-28T20:06:25.988 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-06-28T20:06:26.034 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-28T20:06:26.061 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.43'), ('mon.b', '172.21.15.178'), ('mon.c', '[v2:172.21.15.178:3301,v1:172.21.15.178:6790]')] 2023-06-28T20:06:26.062 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.43,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-06-28T20:06:26.063 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:06:26.063 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-06-28T20:06:26.117 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.43 --add b 172.21.15.178 --addv c '[v2:172.21.15.178:3301,v1:172.21.15.178:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-06-28T20:06:26.207 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-06-28T20:06:26.208 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool: generated fsid b8fd740d-00b9-4308-92a8-48f5fffe8078 2023-06-28T20:06:26.208 INFO:teuthology.orchestra.run.smithi043.stdout:setting min_mon_release = pacific 2023-06-28T20:06:26.208 INFO:teuthology.orchestra.run.smithi043.stdout:epoch 0 2023-06-28T20:06:26.208 INFO:teuthology.orchestra.run.smithi043.stdout:fsid b8fd740d-00b9-4308-92a8-48f5fffe8078 2023-06-28T20:06:26.208 INFO:teuthology.orchestra.run.smithi043.stdout:last_changed 2023-06-28T20:06:26.205413+0000 2023-06-28T20:06:26.208 INFO:teuthology.orchestra.run.smithi043.stdout:created 2023-06-28T20:06:26.205413+0000 2023-06-28T20:06:26.208 INFO:teuthology.orchestra.run.smithi043.stdout:min_mon_release 16 (pacific) 2023-06-28T20:06:26.209 INFO:teuthology.orchestra.run.smithi043.stdout:election_strategy: 1 2023-06-28T20:06:26.209 INFO:teuthology.orchestra.run.smithi043.stdout:0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2023-06-28T20:06:26.209 INFO:teuthology.orchestra.run.smithi043.stdout:1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.b 2023-06-28T20:06:26.209 INFO:teuthology.orchestra.run.smithi043.stdout:2: [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] mon.c 2023-06-28T20:06:26.209 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-06-28T20:06:26.210 DEBUG:teuthology.orchestra.run.smithi043:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-06-28T20:06:26.226 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b8fd740d-00b9-4308-92a8-48f5fffe8078... 2023-06-28T20:06:26.227 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-06-28T20:06:26.268 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-06-28T20:06:26.314 INFO:teuthology.orchestra.run.smithi178.stdout:[global] 2023-06-28T20:06:26.314 INFO:teuthology.orchestra.run.smithi178.stdout: chdir = "" 2023-06-28T20:06:26.314 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-06-28T20:06:26.314 INFO:teuthology.orchestra.run.smithi178.stdout: auth supported = cephx 2023-06-28T20:06:26.314 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.314 INFO:teuthology.orchestra.run.smithi178.stdout: filestore xattr use omap = true 2023-06-28T20:06:26.315 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.315 INFO:teuthology.orchestra.run.smithi178.stdout: mon clock drift allowed = 1.000 2023-06-28T20:06:26.315 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.315 INFO:teuthology.orchestra.run.smithi178.stdout: osd crush chooseleaf type = 0 2023-06-28T20:06:26.315 INFO:teuthology.orchestra.run.smithi178.stdout: auth debug = true 2023-06-28T20:06:26.316 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.316 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on old message = true 2023-06-28T20:06:26.316 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on bug = true 2023-06-28T20:06:26.316 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.316 INFO:teuthology.orchestra.run.smithi178.stdout: mon max pg per osd = 10000 # >= luminous 2023-06-28T20:06:26.316 INFO:teuthology.orchestra.run.smithi178.stdout: mon pg warn max object skew = 0 2023-06-28T20:06:26.316 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.317 INFO:teuthology.orchestra.run.smithi178.stdout: # disable pg_autoscaler by default for new pools 2023-06-28T20:06:26.317 INFO:teuthology.orchestra.run.smithi178.stdout: osd_pool_default_pg_autoscale_mode = off 2023-06-28T20:06:26.317 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.317 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default size = 2 2023-06-28T20:06:26.317 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.317 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow primary affinity = true 2023-06-28T20:06:26.318 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd allow pg remap = true 2023-06-28T20:06:26.318 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on legacy crush tunables = false 2023-06-28T20:06:26.318 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on crush straw calc version zero = false 2023-06-28T20:06:26.318 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on no sortbitwise = false 2023-06-28T20:06:26.318 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on osd down out interval zero = false 2023-06-28T20:06:26.318 INFO:teuthology.orchestra.run.smithi178.stdout: mon warn on too few osds = false 2023-06-28T20:06:26.319 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-06-28T20:06:26.319 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_pool_no_redundancy = false 2023-06-28T20:06:26.319 INFO:teuthology.orchestra.run.smithi178.stdout: mon_allow_pool_size_one = true 2023-06-28T20:06:26.319 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.319 INFO:teuthology.orchestra.run.smithi178.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-06-28T20:06:26.319 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.319 INFO:teuthology.orchestra.run.smithi178.stdout: osd default data pool replay window = 5 2023-06-28T20:06:26.320 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.320 INFO:teuthology.orchestra.run.smithi178.stdout: mon allow pool delete = true 2023-06-28T20:06:26.320 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.320 INFO:teuthology.orchestra.run.smithi178.stdout: mon cluster log file level = debug 2023-06-28T20:06:26.320 INFO:teuthology.orchestra.run.smithi178.stdout: debug asserts on shutdown = true 2023-06-28T20:06:26.320 INFO:teuthology.orchestra.run.smithi178.stdout: mon health detail to clog = false 2023-06-28T20:06:26.320 INFO:teuthology.orchestra.run.smithi178.stdout: mon host = "172.21.15.43,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2023-06-28T20:06:26.320 INFO:teuthology.orchestra.run.smithi178.stdout: ms die on skipped message = False 2023-06-28T20:06:26.321 INFO:teuthology.orchestra.run.smithi178.stdout: fsid = b8fd740d-00b9-4308-92a8-48f5fffe8078 2023-06-28T20:06:26.321 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.321 INFO:teuthology.orchestra.run.smithi178.stdout:[osd] 2023-06-28T20:06:26.321 INFO:teuthology.orchestra.run.smithi178.stdout: osd journal size = 100 2023-06-28T20:06:26.321 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.321 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub load threshold = 5.0 2023-06-28T20:06:26.321 INFO:teuthology.orchestra.run.smithi178.stdout: osd scrub max interval = 600 2023-06-28T20:06:26.321 INFO:teuthology.orchestra.run.smithi178.stdout: osd mclock profile = high_recovery_ops 2023-06-28T20:06:26.322 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.322 INFO:teuthology.orchestra.run.smithi178.stdout: osd recover clone overlap = true 2023-06-28T20:06:26.322 INFO:teuthology.orchestra.run.smithi178.stdout: osd recovery max chunk = 1048576 2023-06-28T20:06:26.322 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.322 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug shutdown = true 2023-06-28T20:06:26.322 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug op order = true 2023-06-28T20:06:26.323 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug verify stray on activate = true 2023-06-28T20:06:26.323 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.323 INFO:teuthology.orchestra.run.smithi178.stdout: osd open classes on start = true 2023-06-28T20:06:26.323 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug pg log writeout = true 2023-06-28T20:06:26.323 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.323 INFO:teuthology.orchestra.run.smithi178.stdout: osd deep scrub update digest min age = 30 2023-06-28T20:06:26.323 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.323 INFO:teuthology.orchestra.run.smithi178.stdout: osd map max advance = 10 2023-06-28T20:06:26.324 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.324 INFO:teuthology.orchestra.run.smithi178.stdout: journal zero on create = true 2023-06-28T20:06:26.324 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.324 INFO:teuthology.orchestra.run.smithi178.stdout: filestore ondisk finisher threads = 3 2023-06-28T20:06:26.324 INFO:teuthology.orchestra.run.smithi178.stdout: filestore apply finisher threads = 3 2023-06-28T20:06:26.324 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.324 INFO:teuthology.orchestra.run.smithi178.stdout: bdev debug aio = true 2023-06-28T20:06:26.325 INFO:teuthology.orchestra.run.smithi178.stdout: osd debug misdirected ops = true 2023-06-28T20:06:26.325 INFO:teuthology.orchestra.run.smithi178.stdout: bdev async discard = True 2023-06-28T20:06:26.325 INFO:teuthology.orchestra.run.smithi178.stdout: bdev enable discard = True 2023-06-28T20:06:26.325 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore allocator = bitmap 2023-06-28T20:06:26.326 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore block size = 96636764160 2023-06-28T20:06:26.326 INFO:teuthology.orchestra.run.smithi178.stdout: bluestore fsck on mount = True 2023-06-28T20:06:26.326 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluefs = 1/20 2023-06-28T20:06:26.326 INFO:teuthology.orchestra.run.smithi178.stdout: debug bluestore = 1/20 2023-06-28T20:06:26.326 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2023-06-28T20:06:26.326 INFO:teuthology.orchestra.run.smithi178.stdout: debug osd = 20 2023-06-28T20:06:26.326 INFO:teuthology.orchestra.run.smithi178.stdout: debug rocksdb = 4/10 2023-06-28T20:06:26.326 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd backfillfull_ratio = 0.85 2023-06-28T20:06:26.327 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd full ratio = 0.9 2023-06-28T20:06:26.327 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd nearfull ratio = 0.8 2023-06-28T20:06:26.327 INFO:teuthology.orchestra.run.smithi178.stdout: osd failsafe full ratio = 0.95 2023-06-28T20:06:26.327 INFO:teuthology.orchestra.run.smithi178.stdout: osd objectstore = bluestore 2023-06-28T20:06:26.327 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2023-06-28T20:06:26.327 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.327 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr] 2023-06-28T20:06:26.327 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2023-06-28T20:06:26.328 INFO:teuthology.orchestra.run.smithi178.stdout: debug mgr = 20 2023-06-28T20:06:26.328 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2023-06-28T20:06:26.328 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2023-06-28T20:06:26.328 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2023-06-28T20:06:26.328 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2023-06-28T20:06:26.329 INFO:teuthology.orchestra.run.smithi178.stdout: mgr/telemetry/nag = false 2023-06-28T20:06:26.329 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.329 INFO:teuthology.orchestra.run.smithi178.stdout:[mon] 2023-06-28T20:06:26.329 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2023-06-28T20:06:26.329 INFO:teuthology.orchestra.run.smithi178.stdout: debug mon = 20 2023-06-28T20:06:26.329 INFO:teuthology.orchestra.run.smithi178.stdout: debug paxos = 20 2023-06-28T20:06:26.329 INFO:teuthology.orchestra.run.smithi178.stdout: debug auth = 20 2023-06-28T20:06:26.329 INFO:teuthology.orchestra.run.smithi178.stdout: mon data avail warn = 5 2023-06-28T20:06:26.330 INFO:teuthology.orchestra.run.smithi178.stdout: mon mgr mkfs grace = 240 2023-06-28T20:06:26.330 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min pgs per osd = 4 2023-06-28T20:06:26.330 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd reporter subtree level = osd 2023-06-28T20:06:26.330 INFO:teuthology.orchestra.run.smithi178.stdout: mon osd prime pg temp = true 2023-06-28T20:06:26.330 INFO:teuthology.orchestra.run.smithi178.stdout: mon reweight min bytes per osd = 10 2023-06-28T20:06:26.330 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.330 INFO:teuthology.orchestra.run.smithi178.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-06-28T20:06:26.330 INFO:teuthology.orchestra.run.smithi178.stdout: auth mon ticket ttl = 660 # 11m 2023-06-28T20:06:26.331 INFO:teuthology.orchestra.run.smithi178.stdout: auth service ticket ttl = 240 # 4m 2023-06-28T20:06:26.331 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.331 INFO:teuthology.orchestra.run.smithi178.stdout: # don't complain about insecure global_id in the test suite 2023-06-28T20:06:26.331 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-06-28T20:06:26.331 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-06-28T20:06:26.331 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.331 INFO:teuthology.orchestra.run.smithi178.stdout: # 1m isn't quite enough 2023-06-28T20:06:26.331 INFO:teuthology.orchestra.run.smithi178.stdout: mon_down_mkfs_grace = 2m 2023-06-28T20:06:26.332 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.332 INFO:teuthology.orchestra.run.smithi178.stdout: mon_warn_on_filestore_osds = false 2023-06-28T20:06:26.332 INFO:teuthology.orchestra.run.smithi178.stdout: mon op complaint time = 120 2023-06-28T20:06:26.332 INFO:teuthology.orchestra.run.smithi178.stdout: 2023-06-28T20:06:26.332 INFO:teuthology.orchestra.run.smithi178.stdout:[client] 2023-06-28T20:06:26.332 INFO:teuthology.orchestra.run.smithi178.stdout: rgw cache enabled = true 2023-06-28T20:06:26.332 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable ops log = true 2023-06-28T20:06:26.333 INFO:teuthology.orchestra.run.smithi178.stdout: rgw enable usage log = true 2023-06-28T20:06:26.333 INFO:teuthology.orchestra.run.smithi178.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-06-28T20:06:26.333 INFO:teuthology.orchestra.run.smithi178.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-06-28T20:06:26.333 INFO:teuthology.orchestra.run.smithi178.stdout: client mount timeout = 600 2023-06-28T20:06:26.333 INFO:teuthology.orchestra.run.smithi178.stdout: debug client = 20 2023-06-28T20:06:26.333 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2023-06-28T20:06:26.333 INFO:teuthology.orchestra.run.smithi178.stdout: pid file = "" 2023-06-28T20:06:26.333 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2023-06-28T20:06:26.334 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2023-06-28T20:06:26.334 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.a] 2023-06-28T20:06:26.334 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.b] 2023-06-28T20:06:26.334 INFO:teuthology.orchestra.run.smithi178.stdout:[mon.c] 2023-06-28T20:06:26.334 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.a] 2023-06-28T20:06:26.334 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.c] 2023-06-28T20:06:26.334 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.b] 2023-06-28T20:06:26.334 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.d] 2023-06-28T20:06:26.335 INFO:teuthology.orchestra.run.smithi178.stdout:[mds] 2023-06-28T20:06:26.335 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds = 20 2023-06-28T20:06:26.335 INFO:teuthology.orchestra.run.smithi178.stdout: debug mds balancer = 20 2023-06-28T20:06:26.335 INFO:teuthology.orchestra.run.smithi178.stdout: debug ms = 1 2023-06-28T20:06:26.335 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug frag = True 2023-06-28T20:06:26.335 INFO:teuthology.orchestra.run.smithi178.stdout: mds debug scatterstat = True 2023-06-28T20:06:26.336 INFO:teuthology.orchestra.run.smithi178.stdout: mds op complaint time = 180 2023-06-28T20:06:26.336 INFO:teuthology.orchestra.run.smithi178.stdout: mds verify scatter = True 2023-06-28T20:06:26.336 INFO:teuthology.orchestra.run.smithi178.stdout: osd op complaint time = 180 2023-06-28T20:06:26.336 INFO:teuthology.orchestra.run.smithi178.stdout: rados mon op timeout = 900 2023-06-28T20:06:26.336 INFO:teuthology.orchestra.run.smithi178.stdout: rados osd op timeout = 900 2023-06-28T20:06:26.337 INFO:teuthology.orchestra.run.smithi043.stdout:[global] 2023-06-28T20:06:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: chdir = "" 2023-06-28T20:06:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-06-28T20:06:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: auth supported = cephx 2023-06-28T20:06:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: filestore xattr use omap = true 2023-06-28T20:06:26.338 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: mon clock drift allowed = 1.000 2023-06-28T20:06:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: osd crush chooseleaf type = 0 2023-06-28T20:06:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: auth debug = true 2023-06-28T20:06:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: ms die on old message = true 2023-06-28T20:06:26.339 INFO:teuthology.orchestra.run.smithi043.stdout: ms die on bug = true 2023-06-28T20:06:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: mon max pg per osd = 10000 # >= luminous 2023-06-28T20:06:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: mon pg warn max object skew = 0 2023-06-28T20:06:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: # disable pg_autoscaler by default for new pools 2023-06-28T20:06:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: osd_pool_default_pg_autoscale_mode = off 2023-06-28T20:06:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.340 INFO:teuthology.orchestra.run.smithi043.stdout: osd pool default size = 2 2023-06-28T20:06:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd allow primary affinity = true 2023-06-28T20:06:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd allow pg remap = true 2023-06-28T20:06:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on legacy crush tunables = false 2023-06-28T20:06:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on crush straw calc version zero = false 2023-06-28T20:06:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on no sortbitwise = false 2023-06-28T20:06:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on osd down out interval zero = false 2023-06-28T20:06:26.341 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on too few osds = false 2023-06-28T20:06:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-06-28T20:06:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_pool_no_redundancy = false 2023-06-28T20:06:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: mon_allow_pool_size_one = true 2023-06-28T20:06:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-06-28T20:06:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: osd default data pool replay window = 5 2023-06-28T20:06:26.342 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: mon allow pool delete = true 2023-06-28T20:06:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: mon cluster log file level = debug 2023-06-28T20:06:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: debug asserts on shutdown = true 2023-06-28T20:06:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: mon health detail to clog = false 2023-06-28T20:06:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: mon host = "172.21.15.43,172.21.15.178,[v2:172.21.15.178:3301,v1:172.21.15.178:6790]" 2023-06-28T20:06:26.343 INFO:teuthology.orchestra.run.smithi043.stdout: ms die on skipped message = False 2023-06-28T20:06:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: fsid = b8fd740d-00b9-4308-92a8-48f5fffe8078 2023-06-28T20:06:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.344 INFO:teuthology.orchestra.run.smithi043.stdout:[osd] 2023-06-28T20:06:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: osd journal size = 100 2023-06-28T20:06:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: osd scrub load threshold = 5.0 2023-06-28T20:06:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: osd scrub max interval = 600 2023-06-28T20:06:26.344 INFO:teuthology.orchestra.run.smithi043.stdout: osd mclock profile = high_recovery_ops 2023-06-28T20:06:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: osd recover clone overlap = true 2023-06-28T20:06:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: osd recovery max chunk = 1048576 2023-06-28T20:06:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug shutdown = true 2023-06-28T20:06:26.345 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug op order = true 2023-06-28T20:06:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug verify stray on activate = true 2023-06-28T20:06:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.346 INFO:teuthology.orchestra.run.smithi043.stdout: osd open classes on start = true 2023-06-28T20:06:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug pg log writeout = true 2023-06-28T20:06:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: osd deep scrub update digest min age = 30 2023-06-28T20:06:26.347 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: osd map max advance = 10 2023-06-28T20:06:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: journal zero on create = true 2023-06-28T20:06:26.348 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: filestore ondisk finisher threads = 3 2023-06-28T20:06:26.349 INFO:teuthology.orchestra.run.smithi043.stdout: filestore apply finisher threads = 3 2023-06-28T20:06:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: bdev debug aio = true 2023-06-28T20:06:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug misdirected ops = true 2023-06-28T20:06:26.350 INFO:teuthology.orchestra.run.smithi043.stdout: bdev async discard = True 2023-06-28T20:06:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: bdev enable discard = True 2023-06-28T20:06:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore allocator = bitmap 2023-06-28T20:06:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore block size = 96636764160 2023-06-28T20:06:26.351 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore fsck on mount = True 2023-06-28T20:06:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: debug bluefs = 1/20 2023-06-28T20:06:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: debug bluestore = 1/20 2023-06-28T20:06:26.352 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2023-06-28T20:06:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: debug osd = 20 2023-06-28T20:06:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: debug rocksdb = 4/10 2023-06-28T20:06:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd backfillfull_ratio = 0.85 2023-06-28T20:06:26.353 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd full ratio = 0.9 2023-06-28T20:06:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd nearfull ratio = 0.8 2023-06-28T20:06:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: osd failsafe full ratio = 0.95 2023-06-28T20:06:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: osd objectstore = bluestore 2023-06-28T20:06:26.354 INFO:teuthology.orchestra.run.smithi043.stdout: osd op complaint time = 180 2023-06-28T20:06:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.355 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr] 2023-06-28T20:06:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2023-06-28T20:06:26.355 INFO:teuthology.orchestra.run.smithi043.stdout: debug mgr = 20 2023-06-28T20:06:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: debug mon = 20 2023-06-28T20:06:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: debug auth = 20 2023-06-28T20:06:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min pgs per osd = 4 2023-06-28T20:06:26.356 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min bytes per osd = 10 2023-06-28T20:06:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: mgr/telemetry/nag = false 2023-06-28T20:06:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.357 INFO:teuthology.orchestra.run.smithi043.stdout:[mon] 2023-06-28T20:06:26.357 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2023-06-28T20:06:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: debug mon = 20 2023-06-28T20:06:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: debug paxos = 20 2023-06-28T20:06:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: debug auth = 20 2023-06-28T20:06:26.358 INFO:teuthology.orchestra.run.smithi043.stdout: mon data avail warn = 5 2023-06-28T20:06:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: mon mgr mkfs grace = 240 2023-06-28T20:06:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min pgs per osd = 4 2023-06-28T20:06:26.359 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd reporter subtree level = osd 2023-06-28T20:06:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd prime pg temp = true 2023-06-28T20:06:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min bytes per osd = 10 2023-06-28T20:06:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-06-28T20:06:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: auth mon ticket ttl = 660 # 11m 2023-06-28T20:06:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: auth service ticket ttl = 240 # 4m 2023-06-28T20:06:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: # don't complain about insecure global_id in the test suite 2023-06-28T20:06:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-06-28T20:06:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-06-28T20:06:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.361 INFO:teuthology.orchestra.run.smithi043.stdout: # 1m isn't quite enough 2023-06-28T20:06:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: mon_down_mkfs_grace = 2m 2023-06-28T20:06:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_filestore_osds = false 2023-06-28T20:06:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: mon op complaint time = 120 2023-06-28T20:06:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:06:26.362 INFO:teuthology.orchestra.run.smithi043.stdout:[client] 2023-06-28T20:06:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: rgw cache enabled = true 2023-06-28T20:06:26.362 INFO:teuthology.orchestra.run.smithi043.stdout: rgw enable ops log = true 2023-06-28T20:06:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: rgw enable usage log = true 2023-06-28T20:06:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-06-28T20:06:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-06-28T20:06:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: client mount timeout = 600 2023-06-28T20:06:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: debug client = 20 2023-06-28T20:06:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2023-06-28T20:06:26.363 INFO:teuthology.orchestra.run.smithi043.stdout: pid file = "" 2023-06-28T20:06:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: rados mon op timeout = 900 2023-06-28T20:06:26.364 INFO:teuthology.orchestra.run.smithi043.stdout: rados osd op timeout = 900 2023-06-28T20:06:26.364 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.a] 2023-06-28T20:06:26.364 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.b] 2023-06-28T20:06:26.364 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.c] 2023-06-28T20:06:26.365 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.a] 2023-06-28T20:06:26.365 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.c] 2023-06-28T20:06:26.365 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.b] 2023-06-28T20:06:26.365 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.d] 2023-06-28T20:06:26.365 INFO:teuthology.orchestra.run.smithi043.stdout:[mds] 2023-06-28T20:06:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: debug mds = 20 2023-06-28T20:06:26.365 INFO:teuthology.orchestra.run.smithi043.stdout: debug mds balancer = 20 2023-06-28T20:06:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2023-06-28T20:06:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: mds debug frag = True 2023-06-28T20:06:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: mds debug scatterstat = True 2023-06-28T20:06:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: mds op complaint time = 180 2023-06-28T20:06:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: mds verify scatter = True 2023-06-28T20:06:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: osd op complaint time = 180 2023-06-28T20:06:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: rados mon op timeout = 900 2023-06-28T20:06:26.366 INFO:teuthology.orchestra.run.smithi043.stdout: rados osd op timeout = 900 2023-06-28T20:06:26.367 INFO:tasks.ceph:Creating admin key on mon.a... 2023-06-28T20:06:26.367 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-06-28T20:06:26.438 INFO:tasks.ceph:Copying monmap to all nodes... 2023-06-28T20:06:26.439 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:06:26.439 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-06-28T20:06:26.455 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:06:26.456 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-06-28T20:06:26.513 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi043.front.sepia.ceph.com 2023-06-28T20:06:26.514 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:06:26.514 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.keyring 2023-06-28T20:06:26.514 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-28T20:06:26.594 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:06:26.594 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-06-28T20:06:26.652 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi178.front.sepia.ceph.com 2023-06-28T20:06:26.653 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:06:26.653 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.keyring 2023-06-28T20:06:26.653 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-28T20:06:26.690 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:06:26.690 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-06-28T20:06:26.747 INFO:tasks.ceph:Setting up mon nodes... 2023-06-28T20:06:26.747 INFO:tasks.ceph:Setting up mgr nodes... 2023-06-28T20:06:26.748 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-06-28T20:06:26.807 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-06-28T20:06:26.810 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-06-28T20:06:26.894 INFO:teuthology.orchestra.run.smithi178.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-06-28T20:06:26.897 INFO:tasks.ceph:Setting up mds nodes... 2023-06-28T20:06:26.898 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-06-28T20:06:26.954 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-06-28T20:06:26.957 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-06-28T20:06:26.983 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-06-28T20:06:27.080 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-06-28T20:06:27.082 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-06-28T20:06:27.110 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-06-28T20:06:27.165 INFO:teuthology.orchestra.run.smithi178.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-06-28T20:06:27.167 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-06-28T20:06:27.194 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-06-28T20:06:27.288 INFO:teuthology.orchestra.run.smithi178.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-06-28T20:06:27.290 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-06-28T20:06:27.316 INFO:tasks.ceph_client:Setting up client nodes... 2023-06-28T20:06:27.317 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-06-28T20:06:27.363 INFO:teuthology.orchestra.run.smithi043.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-06-28T20:06:27.377 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-06-28T20:06:27.421 INFO:teuthology.orchestra.run.smithi043.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-06-28T20:06:27.435 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-06-28T20:06:27.477 INFO:teuthology.orchestra.run.smithi178.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-06-28T20:06:27.490 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-06-28T20:06:27.574 INFO:teuthology.orchestra.run.smithi178.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-06-28T20:06:27.587 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-06-28T20:06:27.588 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi043.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi178.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-06-28T20:06:27.588 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-06-28T20:06:27.616 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-28T20:06:27.616 INFO:tasks.ceph:role: osd.0 2023-06-28T20:06:27.616 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi043.front.sepia.ceph.com 2023-06-28T20:06:27.616 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-06-28T20:06:27.684 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-06-28T20:06:27.684 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-28T20:06:27.684 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-28T20:06:27.684 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-28T20:06:27.684 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-28T20:06:27.684 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2023-06-28T20:06:27.685 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-28T20:06:27.685 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-28T20:06:27.685 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-28T20:06:27.685 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-28T20:06:27.707 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2023-06-28T20:06:27.708 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2023-06-28T20:06:27.709 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-06-28T20:06:27.790 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-06-28T20:06:27.863 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-06-28T20:06:27.930 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-28T20:06:27.930 INFO:tasks.ceph:role: osd.1 2023-06-28T20:06:27.930 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi043.front.sepia.ceph.com 2023-06-28T20:06:27.930 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-06-28T20:06:27.996 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-06-28T20:06:27.996 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-28T20:06:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-28T20:06:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-28T20:06:27.997 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-28T20:06:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2023-06-28T20:06:27.997 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-28T20:06:27.997 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-28T20:06:27.997 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-28T20:06:27.998 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-28T20:06:28.019 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2023-06-28T20:06:28.021 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2023-06-28T20:06:28.021 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-06-28T20:06:28.101 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-06-28T20:06:28.171 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-06-28T20:06:28.238 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-28T20:06:28.239 INFO:tasks.ceph:role: osd.2 2023-06-28T20:06:28.239 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi043.front.sepia.ceph.com 2023-06-28T20:06:28.239 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-06-28T20:06:28.304 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-06-28T20:06:28.305 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-28T20:06:28.305 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-28T20:06:28.305 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-28T20:06:28.305 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-28T20:06:28.305 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2023-06-28T20:06:28.305 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-28T20:06:28.306 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-28T20:06:28.306 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-28T20:06:28.306 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-28T20:06:28.327 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2023-06-28T20:06:28.328 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2023-06-28T20:06:28.329 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-06-28T20:06:28.408 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-06-28T20:06:28.479 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-06-28T20:06:28.545 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-28T20:06:28.545 INFO:tasks.ceph:role: osd.3 2023-06-28T20:06:28.546 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi043.front.sepia.ceph.com 2023-06-28T20:06:28.546 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-06-28T20:06:28.612 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-06-28T20:06:28.612 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-28T20:06:28.612 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-28T20:06:28.613 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-28T20:06:28.613 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-28T20:06:28.613 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2023-06-28T20:06:28.613 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-28T20:06:28.613 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-28T20:06:28.613 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-28T20:06:28.613 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-28T20:06:28.635 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2023-06-28T20:06:28.636 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2023-06-28T20:06:28.637 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-06-28T20:06:28.716 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-06-28T20:06:28.788 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-28T20:06:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:29.002+0000 7f13fc749640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-06-28T20:06:29.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:29.002+0000 7f13fc749640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-06-28T20:06:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:29.002+0000 7f13fc749640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-28T20:06:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:29.002+0000 7f13fc749640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-28T20:06:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:29.002+0000 7f13fc749640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-28T20:06:29.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:29.003+0000 7f13fc749640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-06-28T20:06:31.904 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-06-28T20:06:31.932 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-28T20:06:32.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:32.017+0000 7f426a00a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-06-28T20:06:32.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:32.018+0000 7f426a00a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-06-28T20:06:32.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:32.018+0000 7f426a00a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-28T20:06:32.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:32.018+0000 7f426a00a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-28T20:06:32.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:32.018+0000 7f426a00a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-28T20:06:32.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:32.018+0000 7f426a00a640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-06-28T20:06:34.921 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-06-28T20:06:34.950 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-28T20:06:35.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:35.034+0000 7f9147de4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-06-28T20:06:35.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:35.034+0000 7f9147de4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-06-28T20:06:35.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:35.034+0000 7f9147de4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-28T20:06:35.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:35.034+0000 7f9147de4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-28T20:06:35.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:35.034+0000 7f9147de4640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-28T20:06:35.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:35.034+0000 7f9147de4640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-06-28T20:06:37.931 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-06-28T20:06:37.959 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-28T20:06:38.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:38.044+0000 7fb8cffbf640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-06-28T20:06:38.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:38.044+0000 7fb8cffbf640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-06-28T20:06:38.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:38.044+0000 7fb8cffbf640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-28T20:06:38.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:38.044+0000 7fb8cffbf640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-28T20:06:38.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:38.044+0000 7fb8cffbf640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-28T20:06:38.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:38.044+0000 7fb8cffbf640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-06-28T20:06:40.906 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-06-28T20:06:40.939 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-06-28T20:06:40.971 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-28T20:06:40.972 INFO:tasks.ceph:role: osd.4 2023-06-28T20:06:40.972 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi178.front.sepia.ceph.com 2023-06-28T20:06:40.972 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-06-28T20:06:41.042 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-06-28T20:06:41.042 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-28T20:06:41.042 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-28T20:06:41.043 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-28T20:06:41.043 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-28T20:06:41.043 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2023-06-28T20:06:41.043 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-28T20:06:41.043 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-28T20:06:41.043 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-28T20:06:41.043 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-28T20:06:41.064 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2023-06-28T20:06:41.066 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2023-06-28T20:06:41.066 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-06-28T20:06:41.147 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-06-28T20:06:41.218 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-06-28T20:06:41.285 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-28T20:06:41.285 INFO:tasks.ceph:role: osd.5 2023-06-28T20:06:41.286 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi178.front.sepia.ceph.com 2023-06-28T20:06:41.286 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-06-28T20:06:41.351 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-06-28T20:06:41.351 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-28T20:06:41.352 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-28T20:06:41.352 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-28T20:06:41.352 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-28T20:06:41.352 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2023-06-28T20:06:41.352 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-28T20:06:41.352 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-28T20:06:41.353 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-28T20:06:41.353 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-28T20:06:41.374 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2023-06-28T20:06:41.375 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2023-06-28T20:06:41.376 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-06-28T20:06:41.455 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-06-28T20:06:41.525 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-06-28T20:06:41.591 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-28T20:06:41.592 INFO:tasks.ceph:role: osd.6 2023-06-28T20:06:41.592 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi178.front.sepia.ceph.com 2023-06-28T20:06:41.592 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-06-28T20:06:41.657 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-06-28T20:06:41.657 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-28T20:06:41.658 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-28T20:06:41.658 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-28T20:06:41.658 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-28T20:06:41.658 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2023-06-28T20:06:41.659 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-28T20:06:41.659 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-28T20:06:41.659 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-28T20:06:41.659 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-28T20:06:41.680 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2023-06-28T20:06:41.681 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2023-06-28T20:06:41.681 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-06-28T20:06:41.759 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-06-28T20:06:41.829 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-06-28T20:06:41.895 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-28T20:06:41.895 INFO:tasks.ceph:role: osd.7 2023-06-28T20:06:41.896 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi178.front.sepia.ceph.com 2023-06-28T20:06:41.896 DEBUG:teuthology.orchestra.run.smithi178:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-06-28T20:06:41.961 INFO:teuthology.orchestra.run.smithi178.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-06-28T20:06:41.961 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-28T20:06:41.961 INFO:teuthology.orchestra.run.smithi178.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-28T20:06:41.961 INFO:teuthology.orchestra.run.smithi178.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-28T20:06:41.961 INFO:teuthology.orchestra.run.smithi178.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-28T20:06:41.962 INFO:teuthology.orchestra.run.smithi178.stdout: = sunit=0 swidth=0 blks 2023-06-28T20:06:41.962 INFO:teuthology.orchestra.run.smithi178.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-28T20:06:41.962 INFO:teuthology.orchestra.run.smithi178.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-28T20:06:41.962 INFO:teuthology.orchestra.run.smithi178.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-28T20:06:41.962 INFO:teuthology.orchestra.run.smithi178.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-28T20:06:41.983 INFO:teuthology.orchestra.run.smithi178.stdout:Discarding blocks...Done. 2023-06-28T20:06:41.985 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi178.front.sepia.ceph.com -o noatime 2023-06-28T20:06:41.985 DEBUG:teuthology.orchestra.run.smithi178:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-06-28T20:06:42.065 DEBUG:teuthology.orchestra.run.smithi178:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-06-28T20:06:42.134 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-28T20:06:42.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:42.233+0000 7f960975a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-06-28T20:06:42.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:42.233+0000 7f960975a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-06-28T20:06:42.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:42.233+0000 7f960975a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-28T20:06:42.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:42.233+0000 7f960975a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-28T20:06:42.237 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:42.233+0000 7f960975a640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-28T20:06:42.238 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:42.233+0000 7f960975a640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-06-28T20:06:45.093 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-06-28T20:06:45.124 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-28T20:06:45.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:45.212+0000 7faf5392a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-06-28T20:06:45.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:45.212+0000 7faf5392a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-06-28T20:06:45.216 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:45.212+0000 7faf5392a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-28T20:06:45.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:45.212+0000 7faf5392a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-28T20:06:45.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:45.212+0000 7faf5392a640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-28T20:06:45.217 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:45.212+0000 7faf5392a640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-06-28T20:06:48.088 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-06-28T20:06:48.121 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-28T20:06:48.210 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:48.206+0000 7f444ef59640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-06-28T20:06:48.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:48.207+0000 7f444ef59640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-06-28T20:06:48.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:48.207+0000 7f444ef59640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-28T20:06:48.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:48.207+0000 7f444ef59640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-28T20:06:48.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:48.207+0000 7f444ef59640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-28T20:06:48.211 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:48.207+0000 7f444ef59640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-06-28T20:06:51.071 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-06-28T20:06:51.102 DEBUG:teuthology.orchestra.run.smithi178:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-28T20:06:51.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:51.190+0000 7f2f6a9f5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-06-28T20:06:51.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:51.190+0000 7f2f6a9f5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-06-28T20:06:51.194 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:51.190+0000 7f2f6a9f5640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-28T20:06:51.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:51.190+0000 7f2f6a9f5640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-28T20:06:51.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:51.190+0000 7f2f6a9f5640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-28T20:06:51.195 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:06:51.190+0000 7f2f6a9f5640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-06-28T20:06:54.062 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-06-28T20:06:54.092 INFO:tasks.ceph:Reading keys from all nodes... 2023-06-28T20:06:54.093 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:06:54.093 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-06-28T20:06:54.129 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:06:54.129 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-06-28T20:06:54.200 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:06:54.201 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-06-28T20:06:54.271 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:06:54.271 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-06-28T20:06:54.311 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-28T20:06:54.314 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-28T20:06:54.342 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:06:54.342 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-06-28T20:06:54.410 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:06:54.410 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-06-28T20:06:54.478 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:06:54.479 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-06-28T20:06:54.546 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:06:54.546 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-06-28T20:06:54.577 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:06:54.578 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-06-28T20:06:54.646 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:06:54.647 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-06-28T20:06:54.714 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:06:54.714 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-06-28T20:06:54.780 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:06:54.781 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-06-28T20:06:54.847 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:06:54.847 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-06-28T20:06:54.912 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:06:54.913 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-06-28T20:06:54.978 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:06:54.978 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-06-28T20:06:54.997 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:06:54.998 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-06-28T20:06:55.056 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:06:55.056 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-06-28T20:06:55.073 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:06:55.073 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-06-28T20:06:55.129 INFO:tasks.ceph:Adding keys to all mons... 2023-06-28T20:06:55.129 DEBUG:teuthology.orchestra.run.smithi043:> sudo tee -a /etc/ceph/ceph.keyring 2023-06-28T20:06:55.132 DEBUG:teuthology.orchestra.run.smithi178:> sudo tee -a /etc/ceph/ceph.keyring 2023-06-28T20:06:55.172 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr.y] 2023-06-28T20:06:55.173 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQDCkpxk+Gz8LxAAeucI0skt1Rn3ga6yMBwuzA== 2023-06-28T20:06:55.173 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.a] 2023-06-28T20:06:55.173 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQDCkpxk3XrHOBAAHldrZ+MiPeCvaRDv/Tgx+A== 2023-06-28T20:06:55.173 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.c] 2023-06-28T20:06:55.174 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQDDkpxkScSiBBAAUerAFKjwrjArDAUJ/R9DMg== 2023-06-28T20:06:55.174 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.0] 2023-06-28T20:06:55.174 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQDFkpxkpy8vABAA96KfRPxwK8A6BhwkhzpWSA== 2023-06-28T20:06:55.174 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.1] 2023-06-28T20:06:55.174 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQDIkpxkigYbARAAlX+rtKMQdqlHkBjs5qQllg== 2023-06-28T20:06:55.175 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.2] 2023-06-28T20:06:55.175 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQDLkpxkMJgUAhAAo/lTCTzRKdBb9gkiI5vavQ== 2023-06-28T20:06:55.175 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.3] 2023-06-28T20:06:55.175 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQDOkpxkNbi3AhAABEA96FPfDzl8dRSfBMzv8w== 2023-06-28T20:06:55.176 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr.x] 2023-06-28T20:06:55.176 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQDCkpxkJPYtNRAAoSegj6HmD6qDRmhhE0VNug== 2023-06-28T20:06:55.176 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.b] 2023-06-28T20:06:55.176 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQDDkpxkH7KzCRAAQINxW1rrA8FdxRr3sU0Vjg== 2023-06-28T20:06:55.177 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.d] 2023-06-28T20:06:55.177 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQDDkpxkQ6oIERAAbTYaJNOpl9DlB1mx8KMm/w== 2023-06-28T20:06:55.177 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.4] 2023-06-28T20:06:55.177 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQDSkpxkeYL6DRAA8m/izs8fTALY+tCkH39GGg== 2023-06-28T20:06:55.177 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.5] 2023-06-28T20:06:55.177 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQDVkpxkl+q7DBAAhXlNHW3APq+oCYnzyovY2A== 2023-06-28T20:06:55.177 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.6] 2023-06-28T20:06:55.178 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQDYkpxkUHFoDBAAvV2PKXUosUjwLnzo0bT0zw== 2023-06-28T20:06:55.178 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.7] 2023-06-28T20:06:55.178 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQDbkpxkLYptCxAA8egRdpdmWEozABFeQSao4g== 2023-06-28T20:06:55.178 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2023-06-28T20:06:55.178 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQDDkpxkmceMFRAAjFOucYVKNehSzr5v2Q0vbg== 2023-06-28T20:06:55.178 INFO:teuthology.orchestra.run.smithi043.stdout:[client.2] 2023-06-28T20:06:55.178 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQDDkpxkqMP8GBAAF36pZfz6787IM6vFDAGIqA== 2023-06-28T20:06:55.178 INFO:teuthology.orchestra.run.smithi043.stdout:[client.1] 2023-06-28T20:06:55.179 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQDDkpxkULRaHBAAU3pR+3uFPsVd1kDHh37shg== 2023-06-28T20:06:55.179 INFO:teuthology.orchestra.run.smithi043.stdout:[client.3] 2023-06-28T20:06:55.179 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQDDkpxksb4fIhAASbu90RGfkIMaS2tD7RpFRw== 2023-06-28T20:06:55.194 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr.y] 2023-06-28T20:06:55.194 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDCkpxk+Gz8LxAAeucI0skt1Rn3ga6yMBwuzA== 2023-06-28T20:06:55.195 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.a] 2023-06-28T20:06:55.195 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDCkpxk3XrHOBAAHldrZ+MiPeCvaRDv/Tgx+A== 2023-06-28T20:06:55.195 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.c] 2023-06-28T20:06:55.195 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDDkpxkScSiBBAAUerAFKjwrjArDAUJ/R9DMg== 2023-06-28T20:06:55.195 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.0] 2023-06-28T20:06:55.195 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDFkpxkpy8vABAA96KfRPxwK8A6BhwkhzpWSA== 2023-06-28T20:06:55.195 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.1] 2023-06-28T20:06:55.196 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDIkpxkigYbARAAlX+rtKMQdqlHkBjs5qQllg== 2023-06-28T20:06:55.196 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.2] 2023-06-28T20:06:55.196 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDLkpxkMJgUAhAAo/lTCTzRKdBb9gkiI5vavQ== 2023-06-28T20:06:55.196 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.3] 2023-06-28T20:06:55.196 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDOkpxkNbi3AhAABEA96FPfDzl8dRSfBMzv8w== 2023-06-28T20:06:55.196 INFO:teuthology.orchestra.run.smithi178.stdout:[mgr.x] 2023-06-28T20:06:55.196 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDCkpxkJPYtNRAAoSegj6HmD6qDRmhhE0VNug== 2023-06-28T20:06:55.197 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.b] 2023-06-28T20:06:55.197 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDDkpxkH7KzCRAAQINxW1rrA8FdxRr3sU0Vjg== 2023-06-28T20:06:55.197 INFO:teuthology.orchestra.run.smithi178.stdout:[mds.d] 2023-06-28T20:06:55.197 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDDkpxkQ6oIERAAbTYaJNOpl9DlB1mx8KMm/w== 2023-06-28T20:06:55.197 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.4] 2023-06-28T20:06:55.197 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDSkpxkeYL6DRAA8m/izs8fTALY+tCkH39GGg== 2023-06-28T20:06:55.197 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.5] 2023-06-28T20:06:55.197 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDVkpxkl+q7DBAAhXlNHW3APq+oCYnzyovY2A== 2023-06-28T20:06:55.198 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.6] 2023-06-28T20:06:55.198 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDYkpxkUHFoDBAAvV2PKXUosUjwLnzo0bT0zw== 2023-06-28T20:06:55.198 INFO:teuthology.orchestra.run.smithi178.stdout:[osd.7] 2023-06-28T20:06:55.198 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDbkpxkLYptCxAA8egRdpdmWEozABFeQSao4g== 2023-06-28T20:06:55.198 INFO:teuthology.orchestra.run.smithi178.stdout:[client.0] 2023-06-28T20:06:55.198 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDDkpxkmceMFRAAjFOucYVKNehSzr5v2Q0vbg== 2023-06-28T20:06:55.198 INFO:teuthology.orchestra.run.smithi178.stdout:[client.2] 2023-06-28T20:06:55.199 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDDkpxkqMP8GBAAF36pZfz6787IM6vFDAGIqA== 2023-06-28T20:06:55.199 INFO:teuthology.orchestra.run.smithi178.stdout:[client.1] 2023-06-28T20:06:55.199 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDDkpxkULRaHBAAU3pR+3uFPsVd1kDHh37shg== 2023-06-28T20:06:55.199 INFO:teuthology.orchestra.run.smithi178.stdout:[client.3] 2023-06-28T20:06:55.199 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQDDkpxksb4fIhAASbu90RGfkIMaS2tD7RpFRw== 2023-06-28T20:06:55.200 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-28T20:06:55.222 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-28T20:06:55.293 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-28T20:06:55.295 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-28T20:06:55.352 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-28T20:06:55.355 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-28T20:06:55.411 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-28T20:06:55.413 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-28T20:06:55.465 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-28T20:06:55.468 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-28T20:06:55.520 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-28T20:06:55.522 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-28T20:06:55.575 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-28T20:06:55.577 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-28T20:06:55.629 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-28T20:06:55.631 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-28T20:06:55.683 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-28T20:06:55.686 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-28T20:06:55.742 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-28T20:06:55.746 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-28T20:06:55.801 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-28T20:06:55.803 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-28T20:06:55.856 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-28T20:06:55.859 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-28T20:06:55.912 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-28T20:06:55.914 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-28T20:06:55.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-28T20:06:55.969 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-28T20:06:56.022 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-28T20:06:56.024 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-28T20:06:56.079 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-28T20:06:56.081 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-28T20:06:56.138 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-28T20:06:56.140 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-28T20:06:56.197 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-28T20:06:56.199 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-28T20:06:56.256 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-06-28T20:06:56.256 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-06-28T20:06:56.283 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-28T20:06:56.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-06-28T20:06:56.409 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-06-28T20:06:56.436 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-28T20:06:56.533 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-06-28T20:06:56.559 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-06-28T20:06:56.625 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-28T20:06:56.722 DEBUG:teuthology.orchestra.run.smithi178:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-06-28T20:06:56.749 DEBUG:teuthology.orchestra.run.smithi043:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-06-28T20:06:56.751 DEBUG:teuthology.orchestra.run.smithi178:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-06-28T20:06:56.804 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-06-28T20:06:56.805 INFO:tasks.ceph.mon.a:Restarting daemon 2023-06-28T20:06:56.806 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-06-28T20:06:56.808 INFO:tasks.ceph.mon.a:Started 2023-06-28T20:06:56.808 INFO:tasks.ceph.mon.b:Restarting daemon 2023-06-28T20:06:56.808 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-06-28T20:06:56.848 INFO:tasks.ceph.mon.b:Started 2023-06-28T20:06:56.849 INFO:tasks.ceph.mon.c:Restarting daemon 2023-06-28T20:06:56.849 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-06-28T20:06:56.851 INFO:tasks.ceph.mon.c:Started 2023-06-28T20:06:56.851 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-06-28T20:06:56.852 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-06-28T20:06:56.852 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-06-28T20:06:56.854 INFO:tasks.ceph.mgr.y:Started 2023-06-28T20:06:56.855 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-06-28T20:06:56.855 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-06-28T20:06:56.856 INFO:tasks.ceph.mgr.x:Started 2023-06-28T20:06:56.857 INFO:tasks.ceph:Setting crush tunables to default 2023-06-28T20:06:56.857 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd crush tunables default 2023-06-28T20:06:56.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:56.979+0000 7f086d47b700 1 Processor -- start 2023-06-28T20:06:56.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:56.979+0000 7f086d47b700 1 -- start start 2023-06-28T20:06:56.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:56.980+0000 7f086d47b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 0x7f08680ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:06:56.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:56.980+0000 7f086d47b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086806e320 0x7f08680ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:06:56.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:56.980+0000 7f086d47b700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08680ae950 con 0x7f08680ad7d0 2023-06-28T20:06:56.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:56.980+0000 7f086d47b700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f086815a9c0 con 0x7f086815a5f0 2023-06-28T20:06:56.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:56.980+0000 7f086d47b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08680add40 con 0x7f086806e320 2023-06-28T20:06:56.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:56.980+0000 7f0866ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 0x7f08680ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:06:56.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:56.980+0000 7f08667fc700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f08680ad7d0 0x7f08680aede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41332/0 (socket says 172.21.15.43:41332) 2023-06-28T20:06:56.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:56.980+0000 7f0866ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 0x7f08680ce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52144/0 (socket says 172.21.15.43:52144) 2023-06-28T20:06:56.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:56.980+0000 7f08667fc700 1 -- 172.21.15.43:0/2301039465 learned_addr learned my addr 172.21.15.43:0/2301039465 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:06:56.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:56.980+0000 7f0866ffd700 1 -- 172.21.15.43:0/2301039465 >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 msgr2=0x7f08680ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-06-28T20:06:56.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:56.980+0000 7f0866ffd700 1 -- 172.21.15.43:0/2301039465 >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 msgr2=0x7f08680ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-28T20:06:56.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:56.980+0000 7f0866ffd700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 0x7f08680ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-28T20:06:56.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:56.980+0000 7f08677fe700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086806e320 0x7f08680ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:06:56.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:56.980+0000 7f0866ffd700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 0x7f08680ce7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-28T20:06:56.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:56.980+0000 7f08677fe700 1 -- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086806e320 msgr2=0x7f08680ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-06-28T20:06:56.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:56.980+0000 7f08677fe700 1 -- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086806e320 msgr2=0x7f08680ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-28T20:06:56.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:56.980+0000 7f08677fe700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086806e320 0x7f08680ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-28T20:06:56.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:56.980+0000 7f08677fe700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086806e320 0x7f08680ced10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-28T20:06:57.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:57.181+0000 7f0866ffd700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 0x7f08680ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:06:57.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:57.181+0000 7f0866ffd700 1 -- 172.21.15.43:0/2301039465 >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 msgr2=0x7f08680ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-06-28T20:06:57.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:57.181+0000 7f0866ffd700 1 -- 172.21.15.43:0/2301039465 >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 msgr2=0x7f08680ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-28T20:06:57.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:57.181+0000 7f0866ffd700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 0x7f08680ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-28T20:06:57.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:57.181+0000 7f0866ffd700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 0x7f08680ce7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-28T20:06:57.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:57.181+0000 7f08677fe700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086806e320 0x7f08680ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:06:57.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:57.181+0000 7f08677fe700 1 -- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086806e320 msgr2=0x7f08680ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-06-28T20:06:57.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:57.181+0000 7f08677fe700 1 -- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086806e320 msgr2=0x7f08680ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-28T20:06:57.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:57.181+0000 7f08677fe700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086806e320 0x7f08680ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-28T20:06:57.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:57.181+0000 7f08677fe700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086806e320 0x7f08680ced10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-28T20:06:57.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:57.582+0000 7f0866ffd700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 0x7f08680ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:06:57.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:57.582+0000 7f0866ffd700 1 -- 172.21.15.43:0/2301039465 >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 msgr2=0x7f08680ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-06-28T20:06:57.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:57.582+0000 7f0866ffd700 1 -- 172.21.15.43:0/2301039465 >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 msgr2=0x7f08680ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-28T20:06:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:57.582+0000 7f0866ffd700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 0x7f08680ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-28T20:06:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:57.582+0000 7f0866ffd700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 0x7f08680ce7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-06-28T20:06:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:57.582+0000 7f08677fe700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086806e320 0x7f08680ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:06:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:57.582+0000 7f08677fe700 1 -- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086806e320 msgr2=0x7f08680ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-06-28T20:06:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:57.582+0000 7f08677fe700 1 -- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086806e320 msgr2=0x7f08680ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-28T20:06:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:57.582+0000 7f08677fe700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086806e320 0x7f08680ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-28T20:06:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:57.582+0000 7f08677fe700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086806e320 0x7f08680ced10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-06-28T20:06:58.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:58.383+0000 7f0866ffd700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 0x7f08680ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:06:58.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:58.383+0000 7f0866ffd700 1 -- 172.21.15.43:0/2301039465 >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 msgr2=0x7f08680ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-06-28T20:06:58.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:58.383+0000 7f0866ffd700 1 -- 172.21.15.43:0/2301039465 >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 msgr2=0x7f08680ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-28T20:06:58.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:58.383+0000 7f0866ffd700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 0x7f08680ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-28T20:06:58.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:58.383+0000 7f0866ffd700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 0x7f08680ce7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-06-28T20:06:58.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:58.383+0000 7f08677fe700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086806e320 0x7f08680ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:06:58.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:58.383+0000 7f08677fe700 1 -- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086806e320 msgr2=0x7f08680ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-06-28T20:06:58.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:58.383+0000 7f08677fe700 1 -- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086806e320 msgr2=0x7f08680ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-28T20:06:58.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:58.383+0000 7f08677fe700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086806e320 0x7f08680ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-28T20:06:58.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:58.383+0000 7f08677fe700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086806e320 0x7f08680ced10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-06-28T20:06:59.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:59.981+0000 7f0864ff9700 1 -- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086806e320 msgr2=0x7f08680ced10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-28T20:06:59.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:59.981+0000 7f0864ff9700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f086806e320 0x7f08680ced10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:06:59.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:59.981+0000 7f0864ff9700 1 -- 172.21.15.43:0/2301039465 >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 msgr2=0x7f08680ce7d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-28T20:06:59.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:59.981+0000 7f0864ff9700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.43:3300/0 conn(0x7f086815a5f0 0x7f08680ce7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:06:59.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:59.981+0000 7f0864ff9700 1 -- 172.21.15.43:0/2301039465 >> v1:172.21.15.43:6789/0 conn(0x7f08680ad7d0 legacy=0x7f08680aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:06:59.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:59.981+0000 7f0864ff9700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.178:3300/0 conn(0x7f08480073d0 0x7f0848009880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:06:59.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:59.981+0000 7f0864ff9700 1 -- 172.21.15.43:0/2301039465 --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08680ae020 con 0x7f0848003730 2023-06-28T20:06:59.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:59.981+0000 7f0864ff9700 1 -- 172.21.15.43:0/2301039465 --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f086815a9c0 con 0x7f08480073d0 2023-06-28T20:06:59.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:59.981+0000 7f08667fc700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.178:3300/0 conn(0x7f08480073d0 0x7f0848009880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:06:59.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:59.981+0000 7f08667fc700 1 -- 172.21.15.43:0/2301039465 >> v2:172.21.15.178:3300/0 conn(0x7f08480073d0 msgr2=0x7f0848009880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-28T20:06:59.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:59.981+0000 7f08667fc700 1 -- 172.21.15.43:0/2301039465 >> v2:172.21.15.178:3300/0 conn(0x7f08480073d0 msgr2=0x7f0848009880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-28T20:06:59.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:59.981+0000 7f08667fc700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.178:3300/0 conn(0x7f08480073d0 0x7f0848009880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-28T20:06:59.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:06:59.981+0000 7f08667fc700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.178:3300/0 conn(0x7f08480073d0 0x7f0848009880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-28T20:07:00.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:00.182+0000 7f08667fc700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.178:3300/0 conn(0x7f08480073d0 0x7f0848009880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:00.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:00.182+0000 7f08667fc700 1 -- 172.21.15.43:0/2301039465 >> v2:172.21.15.178:3300/0 conn(0x7f08480073d0 msgr2=0x7f0848009880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-28T20:07:00.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:00.182+0000 7f08667fc700 1 -- 172.21.15.43:0/2301039465 >> v2:172.21.15.178:3300/0 conn(0x7f08480073d0 msgr2=0x7f0848009880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-28T20:07:00.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:00.182+0000 7f08667fc700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.178:3300/0 conn(0x7f08480073d0 0x7f0848009880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-28T20:07:00.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:00.182+0000 7f08667fc700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.178:3300/0 conn(0x7f08480073d0 0x7f0848009880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-28T20:07:00.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:00.583+0000 7f08667fc700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.178:3300/0 conn(0x7f08480073d0 0x7f0848009880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:00.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:00.583+0000 7f08667fc700 1 -- 172.21.15.43:0/2301039465 >> v2:172.21.15.178:3300/0 conn(0x7f08480073d0 msgr2=0x7f0848009880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-28T20:07:00.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:00.583+0000 7f08667fc700 1 -- 172.21.15.43:0/2301039465 >> v2:172.21.15.178:3300/0 conn(0x7f08480073d0 msgr2=0x7f0848009880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-28T20:07:00.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:00.583+0000 7f08667fc700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.178:3300/0 conn(0x7f08480073d0 0x7f0848009880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-28T20:07:00.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:00.583+0000 7f08667fc700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.178:3300/0 conn(0x7f08480073d0 0x7f0848009880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-06-28T20:07:01.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:01.384+0000 7f08667fc700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.178:3300/0 conn(0x7f08480073d0 0x7f0848009880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:01.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:01.384+0000 7f08667fc700 1 -- 172.21.15.43:0/2301039465 >> v2:172.21.15.178:3300/0 conn(0x7f08480073d0 msgr2=0x7f0848009880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-28T20:07:01.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:01.384+0000 7f08667fc700 1 -- 172.21.15.43:0/2301039465 >> v2:172.21.15.178:3300/0 conn(0x7f08480073d0 msgr2=0x7f0848009880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-28T20:07:01.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:01.384+0000 7f08667fc700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.178:3300/0 conn(0x7f08480073d0 0x7f0848009880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-28T20:07:01.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:01.384+0000 7f08667fc700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.178:3300/0 conn(0x7f08480073d0 0x7f0848009880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-06-28T20:07:02.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.177+0000 7f0865ffb700 1 -- 172.21.15.43:0/2301039465 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1658424899 0 0) 0x7f08680ae020 con 0x7f0848003730 2023-06-28T20:07:02.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.177+0000 7f0865ffb700 1 -- 172.21.15.43:0/2301039465 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08540035b0 con 0x7f0848003730 2023-06-28T20:07:02.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.179+0000 7f0865ffb700 1 -- 172.21.15.43:0/2301039465 <== mon.1 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2849080359 0 0) 0x7f08540035b0 con 0x7f0848003730 2023-06-28T20:07:02.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.179+0000 7f0865ffb700 1 -- 172.21.15.43:0/2301039465 >> v2:172.21.15.178:3300/0 conn(0x7f08480073d0 msgr2=0x7f0848009880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-28T20:07:02.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.179+0000 7f0865ffb700 1 --2- 172.21.15.43:0/2301039465 >> v2:172.21.15.178:3300/0 conn(0x7f08480073d0 0x7f0848009880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:02.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.179+0000 7f0865ffb700 1 -- 172.21.15.43:0/2301039465 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084800af80 con 0x7f0848003730 2023-06-28T20:07:02.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.179+0000 7f0865ffb700 1 -- 172.21.15.43:0/2301039465 <== mon.1 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f0860009a50 con 0x7f0848003730 2023-06-28T20:07:02.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.179+0000 7f0865ffb700 1 -- 172.21.15.43:0/2301039465 >> v1:172.21.15.178:6789/0 conn(0x7f0848003730 legacy=0x7f0848005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:02.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.179+0000 7f08677fe700 1 -- 172.21.15.43:0/2301039465 reap_dead start 2023-06-28T20:07:02.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.179+0000 7f0865ffb700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0854003a00 0x7f0854003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:02.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.179+0000 7f0865ffb700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0854004850 0x7f0854004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:02.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.179+0000 7f0865ffb700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0854005320 0x7f085400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:02.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.179+0000 7f0865ffb700 1 -- 172.21.15.43:0/2301039465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086815a9c0 con 0x7f0854004850 2023-06-28T20:07:02.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.179+0000 7f0865ffb700 1 -- 172.21.15.43:0/2301039465 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085400c250 con 0x7f0854005320 2023-06-28T20:07:02.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.179+0000 7f0865ffb700 1 -- 172.21.15.43:0/2301039465 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085400c410 con 0x7f0854003a00 2023-06-28T20:07:02.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.179+0000 7f08667fc700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0854004850 0x7f0854004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:02.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.179+0000 7f0866ffd700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0854003a00 0x7f0854003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:02.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.179+0000 7f08677fe700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0854005320 0x7f085400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:02.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.180+0000 7f0866ffd700 1 -- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0854005320 msgr2=0x7f085400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:02.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.180+0000 7f0866ffd700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0854005320 0x7f085400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:02.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.180+0000 7f0866ffd700 1 -- 172.21.15.43:0/2301039465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0854004850 msgr2=0x7f0854004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:02.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.180+0000 7f0866ffd700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0854004850 0x7f0854004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:02.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.180+0000 7f0866ffd700 1 -- 172.21.15.43:0/2301039465 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08540035b0 con 0x7f0854003a00 2023-06-28T20:07:02.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.180+0000 7f08667fc700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0854004850 0x7f0854004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:02.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.180+0000 7f0866ffd700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0854003a00 0x7f0854003df0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0860009690 tx=0x7f0854005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21ce7590ebd7933e server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:02.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.180+0000 7f0865ffb700 1 -- 172.21.15.43:0/2301039465 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0860003c40 con 0x7f0854003a00 2023-06-28T20:07:02.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.180+0000 7f0865ffb700 1 -- 172.21.15.43:0/2301039465 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0860017470 con 0x7f0854003a00 2023-06-28T20:07:02.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.180+0000 7f0865ffb700 1 -- 172.21.15.43:0/2301039465 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0860017820 con 0x7f0854003a00 2023-06-28T20:07:02.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.181+0000 7f086d47b700 1 -- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0854003a00 msgr2=0x7f0854003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:02.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.181+0000 7f086d47b700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0854003a00 0x7f0854003df0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0860009690 tx=0x7f0854005880 comp rx=0 tx=0).stop 2023-06-28T20:07:02.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.182+0000 7f086d47b700 1 -- 172.21.15.43:0/2301039465 shutdown_connections 2023-06-28T20:07:02.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.182+0000 7f086d47b700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0854005320 0x7f085400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:02.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.182+0000 7f086d47b700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0854004850 0x7f0854004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:02.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.182+0000 7f086d47b700 1 --2- 172.21.15.43:0/2301039465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0854003a00 0x7f0854003df0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:02.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.182+0000 7f086d47b700 1 -- 172.21.15.43:0/2301039465 >> 172.21.15.43:0/2301039465 conn(0x7f08680664b0 msgr2=0x7f08680d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:02.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.182+0000 7f086d47b700 1 -- 172.21.15.43:0/2301039465 shutdown_connections 2023-06-28T20:07:02.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.182+0000 7f086d47b700 1 -- 172.21.15.43:0/2301039465 wait complete. 2023-06-28T20:07:02.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.184+0000 7f086d47b700 1 Processor -- start 2023-06-28T20:07:02.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.184+0000 7f086d47b700 1 -- start start 2023-06-28T20:07:02.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.186+0000 7f086d47b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0854003a00 0x7f08680c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:02.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.186+0000 7f086d47b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0854004850 0x7f08681550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:02.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.186+0000 7f086d47b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0854005320 0x7f08680b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:02.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.186+0000 7f086d47b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08680cdb40 con 0x7f0854003a00 2023-06-28T20:07:02.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.186+0000 7f086d47b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086806b1a0 con 0x7f0854004850 2023-06-28T20:07:02.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.186+0000 7f086d47b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0868155620 con 0x7f0854005320 2023-06-28T20:07:02.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.186+0000 7f0866ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0854003a00 0x7f08680c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:02.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.186+0000 7f0866ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0854003a00 0x7f08680c4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52214/0 (socket says 172.21.15.43:52214) 2023-06-28T20:07:02.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.186+0000 7f08667fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0854004850 0x7f08681550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:02.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.186+0000 7f0866ffd700 1 -- 172.21.15.43:0/1720520846 learned_addr learned my addr 172.21.15.43:0/1720520846 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:02.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.186+0000 7f08677fe700 1 --2- 172.21.15.43:0/1720520846 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0854005320 0x7f08680b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:02.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.186+0000 7f08667fc700 1 -- 172.21.15.43:0/1720520846 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0854005320 msgr2=0x7f08680b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:02.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.186+0000 7f08667fc700 1 --2- 172.21.15.43:0/1720520846 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0854005320 0x7f08680b09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:02.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.186+0000 7f08667fc700 1 -- 172.21.15.43:0/1720520846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0854003a00 msgr2=0x7f08680c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:02.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.186+0000 7f08667fc700 1 --2- 172.21.15.43:0/1720520846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0854003a00 0x7f08680c4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:02.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.186+0000 7f08667fc700 1 -- 172.21.15.43:0/1720520846 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08680cd860 con 0x7f0854004850 2023-06-28T20:07:02.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.187+0000 7f0866ffd700 1 --2- 172.21.15.43:0/1720520846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0854003a00 0x7f08680c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:02.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.187+0000 7f08667fc700 1 --2- 172.21.15.43:0/1720520846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0854004850 0x7f08681550e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0850017680 tx=0x7f0850017990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:02.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.187+0000 7f085f7fe700 1 -- 172.21.15.43:0/1720520846 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0850018650 con 0x7f0854004850 2023-06-28T20:07:02.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.187+0000 7f085f7fe700 1 -- 172.21.15.43:0/1720520846 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08500187b0 con 0x7f0854004850 2023-06-28T20:07:02.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.187+0000 7f086d47b700 1 -- 172.21.15.43:0/1720520846 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f086806b440 con 0x7f0854004850 2023-06-28T20:07:02.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.187+0000 7f085f7fe700 1 -- 172.21.15.43:0/1720520846 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0850018b20 con 0x7f0854004850 2023-06-28T20:07:02.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.187+0000 7f086d47b700 1 -- 172.21.15.43:0/1720520846 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f086806cc60 con 0x7f0854004850 2023-06-28T20:07:02.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.187+0000 7f085f7fe700 1 -- 172.21.15.43:0/1720520846 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f085001e070 con 0x7f0854004850 2023-06-28T20:07:02.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.187+0000 7f085f7fe700 1 -- 172.21.15.43:0/1720520846 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0850021c40 con 0x7f0854004850 2023-06-28T20:07:02.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.189+0000 7f086d47b700 1 -- 172.21.15.43:0/1720520846 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f086806b000 con 0x7f0854004850 2023-06-28T20:07:02.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.191+0000 7f085f7fe700 1 -- 172.21.15.43:0/1720520846 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f086806b000 con 0x7f0854004850 2023-06-28T20:07:02.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.283+0000 7f086d47b700 1 -- 172.21.15.43:0/1720520846 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f08680728b0 con 0x7f0854004850 2023-06-28T20:07:02.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.287+0000 7f085f7fe700 1 -- 172.21.15.43:0/1720520846 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0850026020 con 0x7f0854004850 2023-06-28T20:07:02.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.288+0000 7f086d47b700 1 -- 172.21.15.43:0/1720520846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0854004850 msgr2=0x7f08681550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:02.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.288+0000 7f086d47b700 1 --2- 172.21.15.43:0/1720520846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0854004850 0x7f08681550e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0850017680 tx=0x7f0850017990 comp rx=0 tx=0).stop 2023-06-28T20:07:02.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.288+0000 7f086d47b700 1 -- 172.21.15.43:0/1720520846 shutdown_connections 2023-06-28T20:07:02.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.288+0000 7f086d47b700 1 --2- 172.21.15.43:0/1720520846 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0854005320 0x7f08680b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:02.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.288+0000 7f086d47b700 1 --2- 172.21.15.43:0/1720520846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0854003a00 0x7f08680c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:02.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.288+0000 7f086d47b700 1 --2- 172.21.15.43:0/1720520846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0854004850 0x7f08681550e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:02.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.288+0000 7f086d47b700 1 -- 172.21.15.43:0/1720520846 >> 172.21.15.43:0/1720520846 conn(0x7f08680664b0 msgr2=0x7f0868159660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:02.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.288+0000 7f086d47b700 1 -- 172.21.15.43:0/1720520846 shutdown_connections 2023-06-28T20:07:02.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:02.288+0000 7f086d47b700 1 -- 172.21.15.43:0/1720520846 wait complete. 2023-06-28T20:07:02.293 INFO:teuthology.orchestra.run.smithi043.stderr:adjusted tunables profile to default 2023-06-28T20:07:02.306 INFO:tasks.ceph:check_enable_crimson: False 2023-06-28T20:07:02.306 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-06-28T20:07:02.306 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:07:02.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-06-28T20:07:02.323 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:02.320+0000 7f744426c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-28T20:07:02.335 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:07:02.336 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-06-28T20:07:02.394 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:02.391+0000 7f744426c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-28T20:07:02.404 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:07:02.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-06-28T20:07:02.472 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:07:02.472 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-06-28T20:07:02.475 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:02.472+0000 7f744426c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-28T20:07:02.502 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:07:02.503 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-06-28T20:07:02.535 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:07:02.536 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-06-28T20:07:02.560 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:02.556+0000 7f744426c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-28T20:07:02.605 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:07:02.606 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-06-28T20:07:02.674 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:07:02.675 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-06-28T20:07:02.710 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:02.706+0000 7f744426c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-28T20:07:02.742 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new bd41dbcc-2d80-479d-b0ed-9318bc419195 0 2023-06-28T20:07:02.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.898+0000 7f487bba6700 1 Processor -- start 2023-06-28T20:07:02.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.898+0000 7f487bba6700 1 -- start start 2023-06-28T20:07:02.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.899+0000 7f487bba6700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f48740aec90 0x7f4874160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:02.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.899+0000 7f487bba6700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f48740af060 0x7f4874070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:02.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.899+0000 7f487bba6700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4874160250 con 0x7f487406ccc0 2023-06-28T20:07:02.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.899+0000 7f487bba6700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f487415cb30 con 0x7f48740aec90 2023-06-28T20:07:02.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.899+0000 7f487bba6700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4874071410 con 0x7f48740af060 2023-06-28T20:07:02.904 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.899+0000 7f4879141700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f48740af060 0x7f4874070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:02.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.899+0000 7f4879141700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f48740af060 0x7f4874070bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:47470/0 (socket says 172.21.15.178:47470) 2023-06-28T20:07:02.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.899+0000 7f4879141700 1 -- 172.21.15.178:0/3807216284 learned_addr learned my addr 172.21.15.178:0/3807216284 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-06-28T20:07:02.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.899+0000 7f487a143700 1 --1- 172.21.15.178:0/3807216284 >> v1:172.21.15.178:6789/0 conn(0x7f487406ccc0 0x7f48740b1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:51910/0 (socket says 172.21.15.178:51910) 2023-06-28T20:07:02.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.899+0000 7f4879942700 1 --2- 172.21.15.178:0/3807216284 >> v2:172.21.15.43:3300/0 conn(0x7f48740aec90 0x7f4874160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:02.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.899+0000 7f4879141700 1 -- 172.21.15.178:0/3807216284 >> v2:172.21.15.43:3300/0 conn(0x7f48740aec90 msgr2=0x7f4874160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:02.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.899+0000 7f4879141700 1 --2- 172.21.15.178:0/3807216284 >> v2:172.21.15.43:3300/0 conn(0x7f48740aec90 0x7f4874160cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:02.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.899+0000 7f4879141700 1 -- 172.21.15.178:0/3807216284 >> v1:172.21.15.178:6789/0 conn(0x7f487406ccc0 legacy=0x7f48740b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:02.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.899+0000 7f4879141700 1 -- 172.21.15.178:0/3807216284 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48741604f0 con 0x7f48740af060 2023-06-28T20:07:02.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.899+0000 7f4879141700 1 --2- 172.21.15.178:0/3807216284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48740af060 0x7f4874070bc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f48740d0310 tx=0x7f4864009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c3072d4f0c595bb server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:02.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.899+0000 7f4878940700 1 -- 172.21.15.178:0/3807216284 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4864011830 con 0x7f48740af060 2023-06-28T20:07:02.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.900+0000 7f4878940700 1 -- 172.21.15.178:0/3807216284 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4864011990 con 0x7f48740af060 2023-06-28T20:07:02.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.900+0000 7f4878940700 1 -- 172.21.15.178:0/3807216284 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4864011d20 con 0x7f48740af060 2023-06-28T20:07:02.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.900+0000 7f487bba6700 1 -- 172.21.15.178:0/3807216284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48740af060 msgr2=0x7f4874070bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:02.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.900+0000 7f487bba6700 1 --2- 172.21.15.178:0/3807216284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48740af060 0x7f4874070bc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f48740d0310 tx=0x7f4864009bd0 comp rx=0 tx=0).stop 2023-06-28T20:07:02.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.900+0000 7f487bba6700 1 -- 172.21.15.178:0/3807216284 shutdown_connections 2023-06-28T20:07:02.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.900+0000 7f487bba6700 1 --2- 172.21.15.178:0/3807216284 >> v2:172.21.15.43:3300/0 conn(0x7f48740aec90 0x7f4874160cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:02.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.900+0000 7f487bba6700 1 --2- 172.21.15.178:0/3807216284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48740af060 0x7f4874070bc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:02.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.900+0000 7f487bba6700 1 -- 172.21.15.178:0/3807216284 >> 172.21.15.178:0/3807216284 conn(0x7f487406b000 msgr2=0x7f487406b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:02.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.900+0000 7f487bba6700 1 -- 172.21.15.178:0/3807216284 shutdown_connections 2023-06-28T20:07:02.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.900+0000 7f487bba6700 1 -- 172.21.15.178:0/3807216284 wait complete. 2023-06-28T20:07:02.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.901+0000 7f487bba6700 1 Processor -- start 2023-06-28T20:07:02.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.901+0000 7f487bba6700 1 -- start start 2023-06-28T20:07:02.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.902+0000 7f487bba6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f487406ccc0 0x7f4874150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:02.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.902+0000 7f487bba6700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48740aec90 0x7f48740cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:02.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.902+0000 7f487bba6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48740af060 0x7f48740cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:02.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.902+0000 7f487bba6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48740b4490 con 0x7f48740af060 2023-06-28T20:07:02.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.902+0000 7f487bba6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48740b4a50 con 0x7f487406ccc0 2023-06-28T20:07:02.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.902+0000 7f487bba6700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487415a4b0 con 0x7f48740aec90 2023-06-28T20:07:02.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.902+0000 7f4879942700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f487406ccc0 0x7f4874150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:02.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.902+0000 7f4879141700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48740aec90 0x7f48740cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:02.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.902+0000 7f4879942700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f487406ccc0 0x7f4874150090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:47474/0 (socket says 172.21.15.178:47474) 2023-06-28T20:07:02.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.902+0000 7f4879942700 1 -- 172.21.15.178:0/1966184144 learned_addr learned my addr 172.21.15.178:0/1966184144 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-06-28T20:07:02.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.902+0000 7f487a143700 1 --2- 172.21.15.178:0/1966184144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48740af060 0x7f48740cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:02.913 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.902+0000 7f4879942700 1 -- 172.21.15.178:0/1966184144 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48740aec90 msgr2=0x7f48740cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:02.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.902+0000 7f4879942700 1 --2- 172.21.15.178:0/1966184144 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48740aec90 0x7f48740cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:02.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.902+0000 7f4879942700 1 -- 172.21.15.178:0/1966184144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48740af060 msgr2=0x7f48740cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:02.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.902+0000 7f4879942700 1 --2- 172.21.15.178:0/1966184144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48740af060 0x7f48740cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:02.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.902+0000 7f4879942700 1 -- 172.21.15.178:0/1966184144 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4864009550 con 0x7f487406ccc0 2023-06-28T20:07:02.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.902+0000 7f4879141700 1 --2- 172.21.15.178:0/1966184144 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48740aec90 0x7f48740cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:02.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.903+0000 7f4879942700 1 --2- 172.21.15.178:0/1966184144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f487406ccc0 0x7f4874150090 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f487000e9b0 tx=0x7f487000ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:02.914 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.903+0000 7f486a7fc700 1 -- 172.21.15.178:0/1966184144 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870009710 con 0x7f487406ccc0 2023-06-28T20:07:02.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.903+0000 7f486a7fc700 1 -- 172.21.15.178:0/1966184144 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487000ee50 con 0x7f487406ccc0 2023-06-28T20:07:02.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.903+0000 7f487bba6700 1 -- 172.21.15.178:0/1966184144 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f487415a790 con 0x7f487406ccc0 2023-06-28T20:07:02.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.903+0000 7f486a7fc700 1 -- 172.21.15.178:0/1966184144 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487000ca50 con 0x7f487406ccc0 2023-06-28T20:07:02.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.903+0000 7f487bba6700 1 -- 172.21.15.178:0/1966184144 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48741505d0 con 0x7f487406ccc0 2023-06-28T20:07:02.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.903+0000 7f486a7fc700 1 -- 172.21.15.178:0/1966184144 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4870005a20 con 0x7f487406ccc0 2023-06-28T20:07:02.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.903+0000 7f486a7fc700 1 -- 172.21.15.178:0/1966184144 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f48700137b0 con 0x7f487406ccc0 2023-06-28T20:07:02.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.904+0000 7f487bba6700 1 -- 172.21.15.178:0/1966184144 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f487415a920 con 0x7f487406ccc0 2023-06-28T20:07:02.915 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.906+0000 7f486a7fc700 1 -- 172.21.15.178:0/1966184144 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f4870021060 con 0x7f487406ccc0 2023-06-28T20:07:02.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.989+0000 7f487bba6700 1 -- 172.21.15.178:0/1966184144 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bd41dbcc-2d80-479d-b0ed-9318bc419195", "id": 0} v 0) v1 -- 0x7f48740728b0 con 0x7f487406ccc0 2023-06-28T20:07:02.996 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.993+0000 7f486a7fc700 1 -- 172.21.15.178:0/1966184144 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bd41dbcc-2d80-479d-b0ed-9318bc419195", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4870017070 con 0x7f487406ccc0 2023-06-28T20:07:02.997 INFO:teuthology.orchestra.run.smithi178.stdout:0 2023-06-28T20:07:02.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.994+0000 7f487bba6700 1 -- 172.21.15.178:0/1966184144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f487406ccc0 msgr2=0x7f4874150090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:02.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.994+0000 7f487bba6700 1 --2- 172.21.15.178:0/1966184144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f487406ccc0 0x7f4874150090 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f487000e9b0 tx=0x7f487000ecc0 comp rx=0 tx=0).stop 2023-06-28T20:07:02.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.994+0000 7f487bba6700 1 -- 172.21.15.178:0/1966184144 shutdown_connections 2023-06-28T20:07:02.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.994+0000 7f487bba6700 1 --2- 172.21.15.178:0/1966184144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f48740af060 0x7f48740cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:02.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.994+0000 7f487bba6700 1 --2- 172.21.15.178:0/1966184144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f487406ccc0 0x7f4874150090 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:02.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.994+0000 7f487bba6700 1 --2- 172.21.15.178:0/1966184144 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48740aec90 0x7f48740cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:02.998 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.994+0000 7f487bba6700 1 -- 172.21.15.178:0/1966184144 >> 172.21.15.178:0/1966184144 conn(0x7f487406b000 msgr2=0x7f48740ae0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:02.999 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.994+0000 7f487bba6700 1 -- 172.21.15.178:0/1966184144 shutdown_connections 2023-06-28T20:07:02.999 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:02.994+0000 7f487bba6700 1 -- 172.21.15.178:0/1966184144 wait complete. 2023-06-28T20:07:03.010 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 82402670-b52b-443d-8d53-a63d02145525 1 2023-06-28T20:07:03.053 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:03.049+0000 7f7d8a535200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-28T20:07:03.056 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:03.052+0000 7f744426c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-28T20:07:03.120 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:03.117+0000 7f7d8a535200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-28T20:07:03.130 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.126+0000 7f24e3c22700 1 Processor -- start 2023-06-28T20:07:03.130 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.127+0000 7f24e3c22700 1 -- start start 2023-06-28T20:07:03.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.127+0000 7f24e3c22700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f24dc0cca10 0x7f24dc0ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:03.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.127+0000 7f24e3c22700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f24dc0b1920 0x7f24dc0b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:03.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.127+0000 7f24e3c22700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24dc0b2280 con 0x7f24dc0cf6e0 2023-06-28T20:07:03.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.127+0000 7f24e3c22700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24dc1609c0 con 0x7f24dc0cca10 2023-06-28T20:07:03.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.127+0000 7f24e3c22700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24dc160e90 con 0x7f24dc0b1920 2023-06-28T20:07:03.132 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.128+0000 7f24e21bf700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f24dc0b1920 0x7f24dc0b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:03.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.128+0000 7f24e21bf700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f24dc0b1920 0x7f24dc0b1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:47496/0 (socket says 172.21.15.178:47496) 2023-06-28T20:07:03.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.128+0000 7f24e21bf700 1 -- 172.21.15.178:0/1963851125 learned_addr learned my addr 172.21.15.178:0/1963851125 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-06-28T20:07:03.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.128+0000 7f24e11bd700 1 --2- 172.21.15.178:0/1963851125 >> v2:172.21.15.43:3300/0 conn(0x7f24dc0cca10 0x7f24dc0ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:03.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.128+0000 7f24e21bf700 1 -- 172.21.15.178:0/1963851125 >> v2:172.21.15.43:3300/0 conn(0x7f24dc0cca10 msgr2=0x7f24dc0ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:03.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.128+0000 7f24e21bf700 1 --2- 172.21.15.178:0/1963851125 >> v2:172.21.15.43:3300/0 conn(0x7f24dc0cca10 0x7f24dc0ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.133 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.128+0000 7f24e21bf700 1 -- 172.21.15.178:0/1963851125 >> v1:172.21.15.43:6789/0 conn(0x7f24dc0cf6e0 legacy=0x7f24dc071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:03.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.128+0000 7f24e21bf700 1 -- 172.21.15.178:0/1963851125 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24dc160bb0 con 0x7f24dc0b1920 2023-06-28T20:07:03.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.128+0000 7f24e21bf700 1 --2- 172.21.15.178:0/1963851125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24dc0b1920 0x7f24dc0b1d40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f24dc0aea40 tx=0x7f24d400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d45392b8243ffb78 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:03.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.128+0000 7f24e09bc700 1 -- 172.21.15.178:0/1963851125 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d400e070 con 0x7f24dc0b1920 2023-06-28T20:07:03.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.128+0000 7f24e09bc700 1 -- 172.21.15.178:0/1963851125 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d400b910 con 0x7f24dc0b1920 2023-06-28T20:07:03.134 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.128+0000 7f24e09bc700 1 -- 172.21.15.178:0/1963851125 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d400bcc0 con 0x7f24dc0b1920 2023-06-28T20:07:03.135 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:03.131+0000 7f744426c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-28T20:07:03.137 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.129+0000 7f24e3c22700 1 -- 172.21.15.178:0/1963851125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24dc0b1920 msgr2=0x7f24dc0b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:03.137 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.129+0000 7f24e3c22700 1 --2- 172.21.15.178:0/1963851125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24dc0b1920 0x7f24dc0b1d40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f24dc0aea40 tx=0x7f24d400b330 comp rx=0 tx=0).stop 2023-06-28T20:07:03.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.129+0000 7f24e3c22700 1 -- 172.21.15.178:0/1963851125 shutdown_connections 2023-06-28T20:07:03.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.129+0000 7f24e3c22700 1 --2- 172.21.15.178:0/1963851125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24dc0b1920 0x7f24dc0b1d40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.129+0000 7f24e3c22700 1 --2- 172.21.15.178:0/1963851125 >> v2:172.21.15.43:3300/0 conn(0x7f24dc0cca10 0x7f24dc0ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.129+0000 7f24e3c22700 1 -- 172.21.15.178:0/1963851125 >> 172.21.15.178:0/1963851125 conn(0x7f24dc06b000 msgr2=0x7f24dc06b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:03.138 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.129+0000 7f24e3c22700 1 -- 172.21.15.178:0/1963851125 shutdown_connections 2023-06-28T20:07:03.139 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.129+0000 7f24e3c22700 1 -- 172.21.15.178:0/1963851125 wait complete. 2023-06-28T20:07:03.139 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.130+0000 7f24e3c22700 1 Processor -- start 2023-06-28T20:07:03.139 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.130+0000 7f24e3c22700 1 -- start start 2023-06-28T20:07:03.139 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.131+0000 7f24e3c22700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24dc0b1920 0x7f24dc15aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:03.139 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.131+0000 7f24e3c22700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24dc0cca10 0x7f24dc0b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:03.139 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.131+0000 7f24e3c22700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24dc0cf6e0 0x7f24dc15fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:03.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.131+0000 7f24e3c22700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24dc0b3870 con 0x7f24dc0b1920 2023-06-28T20:07:03.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.131+0000 7f24e3c22700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24dc0b3b50 con 0x7f24dc0cca10 2023-06-28T20:07:03.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.131+0000 7f24e3c22700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24dc0b3e30 con 0x7f24dc0cf6e0 2023-06-28T20:07:03.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.131+0000 7f24e11bd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24dc0cca10 0x7f24dc0b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:03.140 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.131+0000 7f24e21bf700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24dc0cf6e0 0x7f24dc15fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:03.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.131+0000 7f24e11bd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24dc0cca10 0x7f24dc0b29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:47500/0 (socket says 172.21.15.178:47500) 2023-06-28T20:07:03.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.131+0000 7f24e11bd700 1 -- 172.21.15.178:0/2635731756 learned_addr learned my addr 172.21.15.178:0/2635731756 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-06-28T20:07:03.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.131+0000 7f24e19be700 1 --2- 172.21.15.178:0/2635731756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24dc0b1920 0x7f24dc15aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:03.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.131+0000 7f24e11bd700 1 -- 172.21.15.178:0/2635731756 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24dc0cf6e0 msgr2=0x7f24dc15fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:03.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.131+0000 7f24e11bd700 1 --2- 172.21.15.178:0/2635731756 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24dc0cf6e0 0x7f24dc15fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.141 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.131+0000 7f24e11bd700 1 -- 172.21.15.178:0/2635731756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24dc0b1920 msgr2=0x7f24dc15aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.131+0000 7f24e11bd700 1 --2- 172.21.15.178:0/2635731756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24dc0b1920 0x7f24dc15aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.131+0000 7f24e11bd700 1 -- 172.21.15.178:0/2635731756 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d4009b30 con 0x7f24dc0cca10 2023-06-28T20:07:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.131+0000 7f24e21bf700 1 --2- 172.21.15.178:0/2635731756 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24dc0cf6e0 0x7f24dc15fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.131+0000 7f24e11bd700 1 --2- 172.21.15.178:0/2635731756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24dc0cca10 0x7f24dc0b29b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00e9b0 tx=0x7f24cc00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.132+0000 7f24d27fc700 1 -- 172.21.15.178:0/2635731756 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc009710 con 0x7f24dc0cca10 2023-06-28T20:07:03.142 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.132+0000 7f24d27fc700 1 -- 172.21.15.178:0/2635731756 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24cc00ee50 con 0x7f24dc0cca10 2023-06-28T20:07:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.132+0000 7f24e3c22700 1 -- 172.21.15.178:0/2635731756 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24dc0c47a0 con 0x7f24dc0cca10 2023-06-28T20:07:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.132+0000 7f24d27fc700 1 -- 172.21.15.178:0/2635731756 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc00ca50 con 0x7f24dc0cca10 2023-06-28T20:07:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.132+0000 7f24e3c22700 1 -- 172.21.15.178:0/2635731756 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24dc0b3520 con 0x7f24dc0cca10 2023-06-28T20:07:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.132+0000 7f24d27fc700 1 -- 172.21.15.178:0/2635731756 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f24cc005a20 con 0x7f24dc0cca10 2023-06-28T20:07:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.132+0000 7f24d27fc700 1 -- 172.21.15.178:0/2635731756 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f24cc013820 con 0x7f24dc0cca10 2023-06-28T20:07:03.143 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.132+0000 7f24e3c22700 1 -- 172.21.15.178:0/2635731756 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24dc0728b0 con 0x7f24dc0cca10 2023-06-28T20:07:03.144 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.135+0000 7f24d27fc700 1 -- 172.21.15.178:0/2635731756 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f24cc021060 con 0x7f24dc0cca10 2023-06-28T20:07:03.201 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:03.197+0000 7f7d8a535200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-28T20:07:03.223 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.219+0000 7f24e3c22700 1 -- 172.21.15.178:0/2635731756 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd new", "uuid": "82402670-b52b-443d-8d53-a63d02145525", "id": 1} v 0) v1 -- 0x7f24dc15f920 con 0x7f24dc0cca10 2023-06-28T20:07:03.226 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.222+0000 7f24d27fc700 1 -- 172.21.15.178:0/2635731756 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "82402670-b52b-443d-8d53-a63d02145525", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f24cc017070 con 0x7f24dc0cca10 2023-06-28T20:07:03.227 INFO:teuthology.orchestra.run.smithi178.stdout:1 2023-06-28T20:07:03.228 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.223+0000 7f24e3c22700 1 -- 172.21.15.178:0/2635731756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24dc0cca10 msgr2=0x7f24dc0b29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:03.228 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.223+0000 7f24e3c22700 1 --2- 172.21.15.178:0/2635731756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24dc0cca10 0x7f24dc0b29b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00e9b0 tx=0x7f24cc00ecc0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.223+0000 7f24e3c22700 1 -- 172.21.15.178:0/2635731756 shutdown_connections 2023-06-28T20:07:03.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.223+0000 7f24e3c22700 1 --2- 172.21.15.178:0/2635731756 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f24dc0cf6e0 0x7f24dc15fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.223+0000 7f24e3c22700 1 --2- 172.21.15.178:0/2635731756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24dc0b1920 0x7f24dc15aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.223+0000 7f24e3c22700 1 --2- 172.21.15.178:0/2635731756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24dc0cca10 0x7f24dc0b29b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.229 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.223+0000 7f24e3c22700 1 -- 172.21.15.178:0/2635731756 >> 172.21.15.178:0/2635731756 conn(0x7f24dc06b000 msgr2=0x7f24dc0cfab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:03.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.223+0000 7f24e3c22700 1 -- 172.21.15.178:0/2635731756 shutdown_connections 2023-06-28T20:07:03.230 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.223+0000 7f24e3c22700 1 -- 172.21.15.178:0/2635731756 wait complete. 2023-06-28T20:07:03.239 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 27713d73-4e63-4676-af2f-8e6753b8967e 2 2023-06-28T20:07:03.272 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:03.268+0000 7f744426c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-28T20:07:03.282 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:03.279+0000 7f7d8a535200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-28T20:07:03.357 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:03.353+0000 7f744426c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-28T20:07:03.399 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.395+0000 7f59c4842700 1 Processor -- start 2023-06-28T20:07:03.400 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.396+0000 7f59c4842700 1 -- start start 2023-06-28T20:07:03.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.396+0000 7f59c4842700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f59c01592d0 0x7f59c00b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:03.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.396+0000 7f59c4842700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59c0071900 0x7f59c00b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:03.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.396+0000 7f59c4842700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59c0070b40 con 0x7f59c01596a0 2023-06-28T20:07:03.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.396+0000 7f59c4842700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59c0070e20 con 0x7f59c01592d0 2023-06-28T20:07:03.403 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.396+0000 7f59c4842700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c0071100 con 0x7f59c0071900 2023-06-28T20:07:03.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.397+0000 7f59be59c700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f59c01592d0 0x7f59c00b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:03.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.397+0000 7f59bed9d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59c0071900 0x7f59c00b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:03.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.397+0000 7f59be59c700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f59c01592d0 0x7f59c00b19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:47504/0 (socket says 172.21.15.178:47504) 2023-06-28T20:07:03.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.397+0000 7f59bdd9b700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f59c01596a0 0x7f59c015d070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:51946/0 (socket says 172.21.15.178:51946) 2023-06-28T20:07:03.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.397+0000 7f59bed9d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59c0071900 0x7f59c00b1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:45390/0 (socket says 172.21.15.178:45390) 2023-06-28T20:07:03.404 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.397+0000 7f59be59c700 1 -- 172.21.15.178:0/2346226325 learned_addr learned my addr 172.21.15.178:0/2346226325 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-06-28T20:07:03.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.397+0000 7f59be59c700 1 -- 172.21.15.178:0/2346226325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59c0071900 msgr2=0x7f59c00b1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:03.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.397+0000 7f59be59c700 1 --2- 172.21.15.178:0/2346226325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59c0071900 0x7f59c00b1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.397+0000 7f59be59c700 1 -- 172.21.15.178:0/2346226325 >> v1:172.21.15.178:6789/0 conn(0x7f59c01596a0 legacy=0x7f59c015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:03.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.397+0000 7f59bd59a700 1 -- 172.21.15.178:0/2346226325 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2985826113 0 0) 0x7f59c0070b40 con 0x7f59c01596a0 2023-06-28T20:07:03.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.397+0000 7f59be59c700 1 -- 172.21.15.178:0/2346226325 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c00713e0 con 0x7f59c01592d0 2023-06-28T20:07:03.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.397+0000 7f59bed9d700 1 --2- 172.21.15.178:0/2346226325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59c0071900 0x7f59c00b1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:03.405 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.397+0000 7f59be59c700 1 --2- 172.21.15.178:0/2346226325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59c01592d0 0x7f59c00b19d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f59b00098f0 tx=0x7f59b0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86efc5eeb39a9865 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:03.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.397+0000 7f59bd59a700 1 -- 172.21.15.178:0/2346226325 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0011830 con 0x7f59c01592d0 2023-06-28T20:07:03.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.397+0000 7f59bd59a700 1 -- 172.21.15.178:0/2346226325 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b0011990 con 0x7f59c01592d0 2023-06-28T20:07:03.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.397+0000 7f59bd59a700 1 -- 172.21.15.178:0/2346226325 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0011d20 con 0x7f59c01592d0 2023-06-28T20:07:03.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.398+0000 7f59c4842700 1 -- 172.21.15.178:0/2346226325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59c01592d0 msgr2=0x7f59c00b19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:03.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.398+0000 7f59c4842700 1 --2- 172.21.15.178:0/2346226325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59c01592d0 0x7f59c00b19d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f59b00098f0 tx=0x7f59b0009c00 comp rx=0 tx=0).stop 2023-06-28T20:07:03.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.398+0000 7f59c4842700 1 -- 172.21.15.178:0/2346226325 shutdown_connections 2023-06-28T20:07:03.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.398+0000 7f59c4842700 1 --2- 172.21.15.178:0/2346226325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59c01592d0 0x7f59c00b19d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.406 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.398+0000 7f59c4842700 1 --2- 172.21.15.178:0/2346226325 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59c0071900 0x7f59c00b1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.398+0000 7f59c4842700 1 -- 172.21.15.178:0/2346226325 >> 172.21.15.178:0/2346226325 conn(0x7f59c006b000 msgr2=0x7f59c006b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:03.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.398+0000 7f59c4842700 1 -- 172.21.15.178:0/2346226325 shutdown_connections 2023-06-28T20:07:03.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.398+0000 7f59c4842700 1 -- 172.21.15.178:0/2346226325 wait complete. 2023-06-28T20:07:03.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.399+0000 7f59c4842700 1 Processor -- start 2023-06-28T20:07:03.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.400+0000 7f59c4842700 1 -- start start 2023-06-28T20:07:03.409 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.400+0000 7f59c4842700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59c0071900 0x7f59c015b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:03.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.400+0000 7f59c4842700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59c01592d0 0x7f59c0153a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:03.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.400+0000 7f59c4842700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59c01596a0 0x7f59c0143030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:03.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.400+0000 7f59c4842700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c015b560 con 0x7f59c01596a0 2023-06-28T20:07:03.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.400+0000 7f59c4842700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c0153f70 con 0x7f59c0071900 2023-06-28T20:07:03.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.400+0000 7f59c4842700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c015b770 con 0x7f59c01592d0 2023-06-28T20:07:03.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.400+0000 7f59be59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59c0071900 0x7f59c015b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:03.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.400+0000 7f59be59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59c0071900 0x7f59c015b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:47510/0 (socket says 172.21.15.178:47510) 2023-06-28T20:07:03.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.400+0000 7f59be59c700 1 -- 172.21.15.178:0/4198022348 learned_addr learned my addr 172.21.15.178:0/4198022348 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-06-28T20:07:03.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.400+0000 7f59bdd9b700 1 --2- 172.21.15.178:0/4198022348 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59c01592d0 0x7f59c0153a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:03.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.400+0000 7f59bed9d700 1 --2- 172.21.15.178:0/4198022348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59c01596a0 0x7f59c0143030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:03.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.401+0000 7f59be59c700 1 -- 172.21.15.178:0/4198022348 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59c01592d0 msgr2=0x7f59c0153a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:03.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.401+0000 7f59be59c700 1 --2- 172.21.15.178:0/4198022348 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59c01592d0 0x7f59c0153a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.401+0000 7f59be59c700 1 -- 172.21.15.178:0/4198022348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59c01596a0 msgr2=0x7f59c0143030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:03.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.401+0000 7f59be59c700 1 --2- 172.21.15.178:0/4198022348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59c01596a0 0x7f59c0143030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.401+0000 7f59be59c700 1 -- 172.21.15.178:0/4198022348 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b8009a70 con 0x7f59c0071900 2023-06-28T20:07:03.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.401+0000 7f59be59c700 1 --2- 172.21.15.178:0/4198022348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59c0071900 0x7f59c015b020 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009c40 tx=0x7f59b0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:03.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.401+0000 7f59aeffd700 1 -- 172.21.15.178:0/4198022348 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0004400 con 0x7f59c0071900 2023-06-28T20:07:03.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.401+0000 7f59aeffd700 1 -- 172.21.15.178:0/4198022348 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b001f070 con 0x7f59c0071900 2023-06-28T20:07:03.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.401+0000 7f59c4842700 1 -- 172.21.15.178:0/4198022348 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b0009550 con 0x7f59c0071900 2023-06-28T20:07:03.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.401+0000 7f59aeffd700 1 -- 172.21.15.178:0/4198022348 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b00038c0 con 0x7f59c0071900 2023-06-28T20:07:03.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.401+0000 7f59c4842700 1 -- 172.21.15.178:0/4198022348 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c015bd30 con 0x7f59c0071900 2023-06-28T20:07:03.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.401+0000 7f59aeffd700 1 -- 172.21.15.178:0/4198022348 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f59b0021030 con 0x7f59c0071900 2023-06-28T20:07:03.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.401+0000 7f59aeffd700 1 -- 172.21.15.178:0/4198022348 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f59b0024430 con 0x7f59c0071900 2023-06-28T20:07:03.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.402+0000 7f59c4842700 1 -- 172.21.15.178:0/4198022348 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59c00cc3e0 con 0x7f59c0071900 2023-06-28T20:07:03.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.404+0000 7f59aeffd700 1 -- 172.21.15.178:0/4198022348 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f59b0015020 con 0x7f59c0071900 2023-06-28T20:07:03.432 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:03.428+0000 7f7d8a535200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-28T20:07:03.491 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.487+0000 7f59c4842700 1 -- 172.21.15.178:0/4198022348 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd new", "uuid": "27713d73-4e63-4676-af2f-8e6753b8967e", "id": 2} v 0) v1 -- 0x7f59c00728b0 con 0x7f59c0071900 2023-06-28T20:07:03.494 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.491+0000 7f59aeffd700 1 -- 172.21.15.178:0/4198022348 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "27713d73-4e63-4676-af2f-8e6753b8967e", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f59b001a450 con 0x7f59c0071900 2023-06-28T20:07:03.495 INFO:teuthology.orchestra.run.smithi178.stdout:2 2023-06-28T20:07:03.496 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.492+0000 7f59c4842700 1 -- 172.21.15.178:0/4198022348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59c0071900 msgr2=0x7f59c015b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:03.496 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.492+0000 7f59c4842700 1 --2- 172.21.15.178:0/4198022348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59c0071900 0x7f59c015b020 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009c40 tx=0x7f59b0004750 comp rx=0 tx=0).stop 2023-06-28T20:07:03.496 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.492+0000 7f59c4842700 1 -- 172.21.15.178:0/4198022348 shutdown_connections 2023-06-28T20:07:03.496 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.492+0000 7f59c4842700 1 --2- 172.21.15.178:0/4198022348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59c01596a0 0x7f59c0143030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.496 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.492+0000 7f59c4842700 1 --2- 172.21.15.178:0/4198022348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59c0071900 0x7f59c015b020 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.497 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.492+0000 7f59c4842700 1 --2- 172.21.15.178:0/4198022348 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f59c01592d0 0x7f59c0153a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.497 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.492+0000 7f59c4842700 1 -- 172.21.15.178:0/4198022348 >> 172.21.15.178:0/4198022348 conn(0x7f59c006b000 msgr2=0x7f59c0154d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:03.497 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.492+0000 7f59c4842700 1 -- 172.21.15.178:0/4198022348 shutdown_connections 2023-06-28T20:07:03.497 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.492+0000 7f59c4842700 1 -- 172.21.15.178:0/4198022348 wait complete. 2023-06-28T20:07:03.507 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new bcf507c8-d64e-41ee-97ef-102353dbbeeb 3 2023-06-28T20:07:03.667 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.663+0000 7fa91863a700 1 Processor -- start 2023-06-28T20:07:03.668 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.664+0000 7fa91863a700 1 -- start start 2023-06-28T20:07:03.670 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.664+0000 7fa91863a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa9101592d0 0x7fa9100b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:03.670 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.664+0000 7fa91863a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa910071900 0x7fa9100b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:03.670 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.664+0000 7fa91863a700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa910070b40 con 0x7fa9101596a0 2023-06-28T20:07:03.670 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.664+0000 7fa91863a700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa910070e20 con 0x7fa9101592d0 2023-06-28T20:07:03.670 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.664+0000 7fa91863a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa910071100 con 0x7fa910071900 2023-06-28T20:07:03.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.665+0000 7fa915bd5700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fa9101596a0 0x7fa91015d070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:51956/0 (socket says 172.21.15.178:51956) 2023-06-28T20:07:03.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.665+0000 7fa916bd7700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa910071900 0x7fa9100b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:03.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.665+0000 7fa915bd5700 1 -- 172.21.15.178:0/2003647107 learned_addr learned my addr 172.21.15.178:0/2003647107 (peer_addr_for_me v1:172.21.15.178:0/0) 2023-06-28T20:07:03.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.665+0000 7fa9163d6700 1 --2- 172.21.15.178:0/2003647107 >> v2:172.21.15.43:3300/0 conn(0x7fa9101592d0 0x7fa9100b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:03.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.665+0000 7fa9153d4700 1 -- 172.21.15.178:0/2003647107 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3113726745 0 0) 0x7fa910070b40 con 0x7fa9101596a0 2023-06-28T20:07:03.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.665+0000 7fa9153d4700 1 -- 172.21.15.178:0/2003647107 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8fc0035b0 con 0x7fa9101596a0 2023-06-28T20:07:03.671 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.665+0000 7fa916bd7700 1 -- 172.21.15.178:0/2003647107 >> v2:172.21.15.43:3300/0 conn(0x7fa9101592d0 msgr2=0x7fa9100b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:03.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.665+0000 7fa916bd7700 1 --2- 172.21.15.178:0/2003647107 >> v2:172.21.15.43:3300/0 conn(0x7fa9101592d0 0x7fa9100b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.665+0000 7fa916bd7700 1 -- 172.21.15.178:0/2003647107 >> v1:172.21.15.178:6789/0 conn(0x7fa9101596a0 legacy=0x7fa91015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:03.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.665+0000 7fa916bd7700 1 -- 172.21.15.178:0/2003647107 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9100713e0 con 0x7fa910071900 2023-06-28T20:07:03.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.665+0000 7fa916bd7700 1 --2- 172.21.15.178:0/2003647107 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa910071900 0x7fa9100b1f10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa908009d90 tx=0x7fa90800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e62f112a9f72bc9 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:03.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.665+0000 7fa9153d4700 1 -- 172.21.15.178:0/2003647107 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90800e070 con 0x7fa910071900 2023-06-28T20:07:03.672 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.665+0000 7fa9153d4700 1 -- 172.21.15.178:0/2003647107 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90800bb20 con 0x7fa910071900 2023-06-28T20:07:03.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.665+0000 7fa9153d4700 1 -- 172.21.15.178:0/2003647107 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908012430 con 0x7fa910071900 2023-06-28T20:07:03.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.666+0000 7fa91863a700 1 -- 172.21.15.178:0/2003647107 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa910071900 msgr2=0x7fa9100b1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:03.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.666+0000 7fa91863a700 1 --2- 172.21.15.178:0/2003647107 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa910071900 0x7fa9100b1f10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa908009d90 tx=0x7fa90800b330 comp rx=0 tx=0).stop 2023-06-28T20:07:03.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.666+0000 7fa91863a700 1 -- 172.21.15.178:0/2003647107 shutdown_connections 2023-06-28T20:07:03.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.666+0000 7fa91863a700 1 --2- 172.21.15.178:0/2003647107 >> v2:172.21.15.43:3300/0 conn(0x7fa9101592d0 0x7fa9100b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.666+0000 7fa91863a700 1 --2- 172.21.15.178:0/2003647107 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa910071900 0x7fa9100b1f10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.673 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.666+0000 7fa91863a700 1 -- 172.21.15.178:0/2003647107 >> 172.21.15.178:0/2003647107 conn(0x7fa91006b000 msgr2=0x7fa91006b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:03.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.666+0000 7fa91863a700 1 -- 172.21.15.178:0/2003647107 shutdown_connections 2023-06-28T20:07:03.674 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.666+0000 7fa91863a700 1 -- 172.21.15.178:0/2003647107 wait complete. 2023-06-28T20:07:03.676 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.667+0000 7fa91863a700 1 Processor -- start 2023-06-28T20:07:03.676 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.667+0000 7fa91863a700 1 -- start start 2023-06-28T20:07:03.676 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.668+0000 7fa91863a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa910071900 0x7fa9101f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:03.677 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.668+0000 7fa91863a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9101592d0 0x7fa9101f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:03.677 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.668+0000 7fa91863a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9101596a0 0x7fa9101f26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:03.677 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.668+0000 7fa91863a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9101f5030 con 0x7fa910071900 2023-06-28T20:07:03.677 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.668+0000 7fa91863a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9101f3370 con 0x7fa9101592d0 2023-06-28T20:07:03.677 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.668+0000 7fa91863a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9101f2c30 con 0x7fa9101596a0 2023-06-28T20:07:03.677 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.668+0000 7fa915bd5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9101592d0 0x7fa9101f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:03.677 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.668+0000 7fa915bd5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9101592d0 0x7fa9101f2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:47524/0 (socket says 172.21.15.178:47524) 2023-06-28T20:07:03.678 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.668+0000 7fa915bd5700 1 -- 172.21.15.178:0/745119521 learned_addr learned my addr 172.21.15.178:0/745119521 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-06-28T20:07:03.678 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.668+0000 7fa916bd7700 1 --2- 172.21.15.178:0/745119521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9101596a0 0x7fa9101f26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:03.678 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.668+0000 7fa9163d6700 1 --2- 172.21.15.178:0/745119521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa910071900 0x7fa9101f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:03.678 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.668+0000 7fa915bd5700 1 -- 172.21.15.178:0/745119521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9101596a0 msgr2=0x7fa9101f26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:03.678 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.668+0000 7fa915bd5700 1 --2- 172.21.15.178:0/745119521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9101596a0 0x7fa9101f26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.678 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.668+0000 7fa915bd5700 1 -- 172.21.15.178:0/745119521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa910071900 msgr2=0x7fa9101f4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:03.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.668+0000 7fa915bd5700 1 --2- 172.21.15.178:0/745119521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa910071900 0x7fa9101f4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.668+0000 7fa915bd5700 1 -- 172.21.15.178:0/745119521 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa908009a70 con 0x7fa9101592d0 2023-06-28T20:07:03.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.668+0000 7fa916bd7700 1 --2- 172.21.15.178:0/745119521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9101596a0 0x7fa9101f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:03.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.669+0000 7fa915bd5700 1 --2- 172.21.15.178:0/745119521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9101592d0 0x7fa9101f2e30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa900005910 tx=0x7fa90000c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:03.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.669+0000 7fa906ffd700 1 -- 172.21.15.178:0/745119521 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa900005d70 con 0x7fa9101592d0 2023-06-28T20:07:03.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.669+0000 7fa906ffd700 1 -- 172.21.15.178:0/745119521 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90000ce80 con 0x7fa9101592d0 2023-06-28T20:07:03.679 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.669+0000 7fa91863a700 1 -- 172.21.15.178:0/745119521 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9100acc10 con 0x7fa9101592d0 2023-06-28T20:07:03.680 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.669+0000 7fa906ffd700 1 -- 172.21.15.178:0/745119521 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9000072f0 con 0x7fa9101592d0 2023-06-28T20:07:03.680 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.669+0000 7fa91863a700 1 -- 172.21.15.178:0/745119521 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9100ad1d0 con 0x7fa9101592d0 2023-06-28T20:07:03.680 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.669+0000 7fa906ffd700 1 -- 172.21.15.178:0/745119521 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa90001c440 con 0x7fa9101592d0 2023-06-28T20:07:03.680 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.669+0000 7fa906ffd700 1 -- 172.21.15.178:0/745119521 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fa9000077a0 con 0x7fa9101592d0 2023-06-28T20:07:03.680 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.670+0000 7fa91863a700 1 -- 172.21.15.178:0/745119521 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9100cc3e0 con 0x7fa9101592d0 2023-06-28T20:07:03.680 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.672+0000 7fa906ffd700 1 -- 172.21.15.178:0/745119521 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fa900017070 con 0x7fa9101592d0 2023-06-28T20:07:03.759 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.755+0000 7fa91863a700 1 -- 172.21.15.178:0/745119521 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bcf507c8-d64e-41ee-97ef-102353dbbeeb", "id": 3} v 0) v1 -- 0x7fa9100728b0 con 0x7fa9101592d0 2023-06-28T20:07:03.762 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.759+0000 7fa906ffd700 1 -- 172.21.15.178:0/745119521 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bcf507c8-d64e-41ee-97ef-102353dbbeeb", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa90001c8d0 con 0x7fa9101592d0 2023-06-28T20:07:03.762 INFO:teuthology.orchestra.run.smithi178.stdout:3 2023-06-28T20:07:03.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.759+0000 7fa91863a700 1 -- 172.21.15.178:0/745119521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9101592d0 msgr2=0x7fa9101f2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:03.763 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.759+0000 7fa91863a700 1 --2- 172.21.15.178:0/745119521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9101592d0 0x7fa9101f2e30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa900005910 tx=0x7fa90000c7b0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.759+0000 7fa91863a700 1 -- 172.21.15.178:0/745119521 shutdown_connections 2023-06-28T20:07:03.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.759+0000 7fa91863a700 1 --2- 172.21.15.178:0/745119521 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa9101596a0 0x7fa9101f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.759+0000 7fa91863a700 1 --2- 172.21.15.178:0/745119521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa910071900 0x7fa9101f4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.759+0000 7fa91863a700 1 --2- 172.21.15.178:0/745119521 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9101592d0 0x7fa9101f2e30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.759+0000 7fa91863a700 1 -- 172.21.15.178:0/745119521 >> 172.21.15.178:0/745119521 conn(0x7fa91006b000 msgr2=0x7fa9100cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:03.764 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.759+0000 7fa91863a700 1 -- 172.21.15.178:0/745119521 shutdown_connections 2023-06-28T20:07:03.765 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.759+0000 7fa91863a700 1 -- 172.21.15.178:0/745119521 wait complete. 2023-06-28T20:07:03.774 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new 97b48458-4459-43be-bc4a-8e5100067cd0 4 2023-06-28T20:07:03.792 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:03.789+0000 7f7d8a535200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-28T20:07:03.811 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:03.806+0000 7f744426c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-28T20:07:03.872 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:03.868+0000 7f7d8a535200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-28T20:07:03.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.892+0000 7f53017e9700 1 Processor -- start 2023-06-28T20:07:03.896 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.892+0000 7f53017e9700 1 -- start start 2023-06-28T20:07:03.898 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.893+0000 7f53017e9700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f52fc15a5f0 0x7f52fc0ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:03.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.893+0000 7f53017e9700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52fc0ad7d0 0x7f52fc0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:03.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.893+0000 7f53017e9700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52fc0ae950 con 0x7f52fc0aede0 2023-06-28T20:07:03.899 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.893+0000 7f53017e9700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52fc15a9c0 con 0x7f52fc15a5f0 2023-06-28T20:07:03.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.893+0000 7f53017e9700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc0add40 con 0x7f52fc0ad7d0 2023-06-28T20:07:03.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.893+0000 7f52faffd700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f52fc15a5f0 0x7f52fc0ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:03.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.893+0000 7f52fb7fe700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f52fc0aede0 0x7f52fc0af230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:51968/0 (socket says 172.21.15.178:51968) 2023-06-28T20:07:03.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.893+0000 7f52fb7fe700 1 -- 172.21.15.178:0/3730573000 learned_addr learned my addr 172.21.15.178:0/3730573000 (peer_addr_for_me v1:172.21.15.178:0/0) 2023-06-28T20:07:03.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.893+0000 7f52fa7fc700 1 --2- 172.21.15.178:0/3730573000 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52fc0ad7d0 0x7f52fc0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:03.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.894+0000 7f52faffd700 1 -- 172.21.15.178:0/3730573000 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52fc0ad7d0 msgr2=0x7f52fc0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:03.900 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.894+0000 7f52faffd700 1 --2- 172.21.15.178:0/3730573000 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52fc0ad7d0 0x7f52fc0ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.894+0000 7f52faffd700 1 -- 172.21.15.178:0/3730573000 >> v1:172.21.15.178:6789/0 conn(0x7f52fc0aede0 legacy=0x7f52fc0af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:03.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.894+0000 7f52faffd700 1 -- 172.21.15.178:0/3730573000 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52fc0ae020 con 0x7f52fc15a5f0 2023-06-28T20:07:03.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.894+0000 7f52f9ffb700 1 -- 172.21.15.178:0/3730573000 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2575389799 0 0) 0x7f52fc0ae950 con 0x7f52fc0aede0 2023-06-28T20:07:03.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.894+0000 7f52faffd700 1 --2- 172.21.15.178:0/3730573000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52fc15a5f0 0x7f52fc0ce7d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009820 tx=0x7f52f0009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65f844a304e4f0f5 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:03.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.894+0000 7f52f9ffb700 1 -- 172.21.15.178:0/3730573000 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f0005e30 con 0x7f52fc15a5f0 2023-06-28T20:07:03.901 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.894+0000 7f52f9ffb700 1 -- 172.21.15.178:0/3730573000 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f0006030 con 0x7f52fc15a5f0 2023-06-28T20:07:03.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.894+0000 7f52f9ffb700 1 -- 172.21.15.178:0/3730573000 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f00063c0 con 0x7f52fc15a5f0 2023-06-28T20:07:03.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.895+0000 7f53017e9700 1 -- 172.21.15.178:0/3730573000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52fc15a5f0 msgr2=0x7f52fc0ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:03.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.895+0000 7f53017e9700 1 --2- 172.21.15.178:0/3730573000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52fc15a5f0 0x7f52fc0ce7d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009820 tx=0x7f52f0009b30 comp rx=0 tx=0).stop 2023-06-28T20:07:03.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.895+0000 7f53017e9700 1 -- 172.21.15.178:0/3730573000 shutdown_connections 2023-06-28T20:07:03.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.895+0000 7f53017e9700 1 --2- 172.21.15.178:0/3730573000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52fc15a5f0 0x7f52fc0ce7d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.902 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.895+0000 7f53017e9700 1 --2- 172.21.15.178:0/3730573000 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52fc0ad7d0 0x7f52fc0ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.895+0000 7f53017e9700 1 -- 172.21.15.178:0/3730573000 >> 172.21.15.178:0/3730573000 conn(0x7f52fc0664b0 msgr2=0x7f52fc0d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:03.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.895+0000 7f53017e9700 1 -- 172.21.15.178:0/3730573000 shutdown_connections 2023-06-28T20:07:03.903 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.895+0000 7f53017e9700 1 -- 172.21.15.178:0/3730573000 wait complete. 2023-06-28T20:07:03.905 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.896+0000 7f53017e9700 1 Processor -- start 2023-06-28T20:07:03.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.896+0000 7f53017e9700 1 -- start start 2023-06-28T20:07:03.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.897+0000 7f53017e9700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52fc0ad7d0 0x7f52fc1f3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:03.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.897+0000 7f53017e9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52fc0aede0 0x7f52fc1e3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:03.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.897+0000 7f53017e9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52fc15a5f0 0x7f52fc0c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:03.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.897+0000 7f53017e9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc1f31b0 con 0x7f52fc15a5f0 2023-06-28T20:07:03.906 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.897+0000 7f53017e9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc1e48a0 con 0x7f52fc0aede0 2023-06-28T20:07:03.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.897+0000 7f53017e9700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc1f3420 con 0x7f52fc0ad7d0 2023-06-28T20:07:03.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.897+0000 7f52fa7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52fc0aede0 0x7f52fc1e3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:03.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.897+0000 7f52faffd700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52fc0ad7d0 0x7f52fc1f3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:03.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.897+0000 7f52fa7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52fc0aede0 0x7f52fc1e3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:47534/0 (socket says 172.21.15.178:47534) 2023-06-28T20:07:03.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.897+0000 7f52faffd700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52fc0ad7d0 0x7f52fc1f3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:45414/0 (socket says 172.21.15.178:45414) 2023-06-28T20:07:03.907 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.897+0000 7f52fa7fc700 1 -- 172.21.15.178:0/650713017 learned_addr learned my addr 172.21.15.178:0/650713017 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-06-28T20:07:03.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.897+0000 7f52fb7fe700 1 --2- 172.21.15.178:0/650713017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52fc15a5f0 0x7f52fc0c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:03.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.897+0000 7f52fa7fc700 1 -- 172.21.15.178:0/650713017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52fc0ad7d0 msgr2=0x7f52fc1f3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:03.908 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.897+0000 7f52fa7fc700 1 --2- 172.21.15.178:0/650713017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52fc0ad7d0 0x7f52fc1f3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.897+0000 7f52fa7fc700 1 -- 172.21.15.178:0/650713017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52fc15a5f0 msgr2=0x7f52fc0c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:03.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.897+0000 7f52fa7fc700 1 --2- 172.21.15.178:0/650713017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52fc15a5f0 0x7f52fc0c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.909 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.897+0000 7f52fa7fc700 1 -- 172.21.15.178:0/650713017 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f0009430 con 0x7f52fc0aede0 2023-06-28T20:07:03.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.897+0000 7f52faffd700 1 --2- 172.21.15.178:0/650713017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52fc0ad7d0 0x7f52fc1f3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:03.910 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.898+0000 7f52fa7fc700 1 --2- 172.21.15.178:0/650713017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52fc0aede0 0x7f52fc1e3ea0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00e9b0 tx=0x7f52ec00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:03.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.898+0000 7f52e37fe700 1 -- 172.21.15.178:0/650713017 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec009710 con 0x7f52fc0aede0 2023-06-28T20:07:03.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.898+0000 7f52e37fe700 1 -- 172.21.15.178:0/650713017 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ec00ee50 con 0x7f52fc0aede0 2023-06-28T20:07:03.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.898+0000 7f53017e9700 1 -- 172.21.15.178:0/650713017 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52fc1f3700 con 0x7f52fc0aede0 2023-06-28T20:07:03.911 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.898+0000 7f52e37fe700 1 -- 172.21.15.178:0/650713017 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec00ca50 con 0x7f52fc0aede0 2023-06-28T20:07:03.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.898+0000 7f53017e9700 1 -- 172.21.15.178:0/650713017 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52fc1e43e0 con 0x7f52fc0aede0 2023-06-28T20:07:03.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.898+0000 7f52e37fe700 1 -- 172.21.15.178:0/650713017 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f52ec005a20 con 0x7f52fc0aede0 2023-06-28T20:07:03.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.898+0000 7f52e37fe700 1 -- 172.21.15.178:0/650713017 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f52ec006700 con 0x7f52fc0aede0 2023-06-28T20:07:03.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.899+0000 7f53017e9700 1 -- 172.21.15.178:0/650713017 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52fc1e4660 con 0x7f52fc0aede0 2023-06-28T20:07:03.912 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.901+0000 7f52e37fe700 1 -- 172.21.15.178:0/650713017 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f52fc1e4660 con 0x7f52fc0aede0 2023-06-28T20:07:03.990 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.984+0000 7f53017e9700 1 -- 172.21.15.178:0/650713017 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd new", "uuid": "97b48458-4459-43be-bc4a-8e5100067cd0", "id": 4} v 0) v1 -- 0x7f52fc0728b0 con 0x7f52fc0aede0 2023-06-28T20:07:03.991 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.988+0000 7f52e37fe700 1 -- 172.21.15.178:0/650713017 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "97b48458-4459-43be-bc4a-8e5100067cd0", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f52ec021060 con 0x7f52fc0aede0 2023-06-28T20:07:03.992 INFO:teuthology.orchestra.run.smithi178.stdout:4 2023-06-28T20:07:03.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.989+0000 7f53017e9700 1 -- 172.21.15.178:0/650713017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52fc0aede0 msgr2=0x7f52fc1e3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:03.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.989+0000 7f53017e9700 1 --2- 172.21.15.178:0/650713017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52fc0aede0 0x7f52fc1e3ea0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00e9b0 tx=0x7f52ec00ecc0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.989+0000 7f53017e9700 1 -- 172.21.15.178:0/650713017 shutdown_connections 2023-06-28T20:07:03.993 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.989+0000 7f53017e9700 1 --2- 172.21.15.178:0/650713017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52fc15a5f0 0x7f52fc0c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.994 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.989+0000 7f53017e9700 1 --2- 172.21.15.178:0/650713017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52fc0ad7d0 0x7f52fc1f3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.994 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.989+0000 7f53017e9700 1 --2- 172.21.15.178:0/650713017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52fc0aede0 0x7f52fc1e3ea0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:03.994 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.989+0000 7f53017e9700 1 -- 172.21.15.178:0/650713017 >> 172.21.15.178:0/650713017 conn(0x7f52fc0664b0 msgr2=0x7f52fc159720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:03.994 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.989+0000 7f53017e9700 1 -- 172.21.15.178:0/650713017 shutdown_connections 2023-06-28T20:07:03.994 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:03.989+0000 7f53017e9700 1 -- 172.21.15.178:0/650713017 wait complete. 2023-06-28T20:07:04.004 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new b9d323dd-4ed4-4710-8eb6-713385fafe44 5 2023-06-28T20:07:04.005 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:04.001+0000 7f7d8a535200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-28T20:07:04.090 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:04.087+0000 7f7d8a535200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-28T20:07:04.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.164+0000 7f3a4f820700 1 Processor -- start 2023-06-28T20:07:04.168 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.164+0000 7f3a4f820700 1 -- start start 2023-06-28T20:07:04.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.165+0000 7f3a4f820700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3a480afdf0 0x7f3a4806da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:04.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.165+0000 7f3a4f820700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a480ae950 con 0x7f3a4815a5f0 2023-06-28T20:07:04.169 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.165+0000 7f3a4f820700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a4815a9c0 con 0x7f3a480ceee0 2023-06-28T20:07:04.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.165+0000 7f3a4f820700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a480add40 con 0x7f3a480afdf0 2023-06-28T20:07:04.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.165+0000 7f3a4ddbd700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3a480afdf0 0x7f3a4806da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:04.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.165+0000 7f3a4cdbb700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f3a480ceee0 0x7f3a4806d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.178:51980/0 (socket says 172.21.15.178:51980) 2023-06-28T20:07:04.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.165+0000 7f3a4cdbb700 1 -- 172.21.15.178:0/2873924622 learned_addr learned my addr 172.21.15.178:0/2873924622 (peer_addr_for_me v1:172.21.15.178:0/0) 2023-06-28T20:07:04.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.165+0000 7f3a4ddbd700 1 -- 172.21.15.178:0/2873924622 >> v1:172.21.15.178:6789/0 conn(0x7f3a480ceee0 legacy=0x7f3a4806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:04.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.165+0000 7f3a4ddbd700 1 -- 172.21.15.178:0/2873924622 >> v1:172.21.15.43:6789/0 conn(0x7f3a4815a5f0 legacy=0x7f3a480ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:04.170 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.165+0000 7f3a4ddbd700 1 -- 172.21.15.178:0/2873924622 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a480ae020 con 0x7f3a480afdf0 2023-06-28T20:07:04.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.166+0000 7f3a4ddbd700 1 --2- 172.21.15.178:0/2873924622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a480afdf0 0x7f3a4806da20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3a40009d90 tx=0x7f3a4000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c3bae8a73931b36 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:04.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.166+0000 7f3a3ffff700 1 -- 172.21.15.178:0/2873924622 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4000e070 con 0x7f3a480afdf0 2023-06-28T20:07:04.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.166+0000 7f3a3ffff700 1 -- 172.21.15.178:0/2873924622 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a4000bbd0 con 0x7f3a480afdf0 2023-06-28T20:07:04.171 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.166+0000 7f3a3ffff700 1 -- 172.21.15.178:0/2873924622 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a40012430 con 0x7f3a480afdf0 2023-06-28T20:07:04.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.166+0000 7f3a4f820700 1 -- 172.21.15.178:0/2873924622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a480afdf0 msgr2=0x7f3a4806da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:04.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.166+0000 7f3a4f820700 1 --2- 172.21.15.178:0/2873924622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a480afdf0 0x7f3a4806da20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3a40009d90 tx=0x7f3a4000b330 comp rx=0 tx=0).stop 2023-06-28T20:07:04.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.166+0000 7f3a4f820700 1 -- 172.21.15.178:0/2873924622 shutdown_connections 2023-06-28T20:07:04.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.166+0000 7f3a4f820700 1 --2- 172.21.15.178:0/2873924622 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a480afdf0 0x7f3a4806da20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.166+0000 7f3a4f820700 1 -- 172.21.15.178:0/2873924622 >> 172.21.15.178:0/2873924622 conn(0x7f3a480664b0 msgr2=0x7f3a480d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:04.172 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.166+0000 7f3a4f820700 1 -- 172.21.15.178:0/2873924622 shutdown_connections 2023-06-28T20:07:04.173 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.166+0000 7f3a4f820700 1 -- 172.21.15.178:0/2873924622 wait complete. 2023-06-28T20:07:04.174 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.168+0000 7f3a4f820700 1 Processor -- start 2023-06-28T20:07:04.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.168+0000 7f3a4f820700 1 -- start start 2023-06-28T20:07:04.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.169+0000 7f3a4f820700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a480afdf0 0x7f3a481f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:04.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.169+0000 7f3a4f820700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a480ceee0 0x7f3a480b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:04.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.169+0000 7f3a4f820700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a4815a5f0 0x7f3a481517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:04.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.169+0000 7f3a4f820700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a481f3620 con 0x7f3a480ceee0 2023-06-28T20:07:04.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.169+0000 7f3a4f820700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a48152710 con 0x7f3a4815a5f0 2023-06-28T20:07:04.175 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.169+0000 7f3a4f820700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a480c5670 con 0x7f3a480afdf0 2023-06-28T20:07:04.176 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.169+0000 7f3a4ddbd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a4815a5f0 0x7f3a481517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:04.176 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.169+0000 7f3a4d5bc700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a480afdf0 0x7f3a481f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:04.176 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.169+0000 7f3a4d5bc700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a480afdf0 0x7f3a481f30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.178:45426/0 (socket says 172.21.15.178:45426) 2023-06-28T20:07:04.176 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.169+0000 7f3a4d5bc700 1 -- 172.21.15.178:0/1366807302 learned_addr learned my addr 172.21.15.178:0/1366807302 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-06-28T20:07:04.176 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.169+0000 7f3a4ddbd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a4815a5f0 0x7f3a481517c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:47548/0 (socket says 172.21.15.178:47548) 2023-06-28T20:07:04.176 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.169+0000 7f3a4cdbb700 1 --2- 172.21.15.178:0/1366807302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a480ceee0 0x7f3a480b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:04.176 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.169+0000 7f3a4ddbd700 1 -- 172.21.15.178:0/1366807302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a480afdf0 msgr2=0x7f3a481f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:04.177 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.169+0000 7f3a4ddbd700 1 --2- 172.21.15.178:0/1366807302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a480afdf0 0x7f3a481f30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.177 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.169+0000 7f3a4ddbd700 1 -- 172.21.15.178:0/1366807302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a480ceee0 msgr2=0x7f3a480b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:04.177 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.169+0000 7f3a4ddbd700 1 --2- 172.21.15.178:0/1366807302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a480ceee0 0x7f3a480b1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.177 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.169+0000 7f3a4ddbd700 1 -- 172.21.15.178:0/1366807302 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a40009a70 con 0x7f3a4815a5f0 2023-06-28T20:07:04.177 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.169+0000 7f3a4d5bc700 1 --2- 172.21.15.178:0/1366807302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a480afdf0 0x7f3a481f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:04.177 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.170+0000 7f3a4ddbd700 1 --2- 172.21.15.178:0/1366807302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a4815a5f0 0x7f3a481517c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3a40016000 tx=0x7f3a40004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:04.177 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.170+0000 7f3a3dffb700 1 -- 172.21.15.178:0/1366807302 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a40012da0 con 0x7f3a4815a5f0 2023-06-28T20:07:04.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.170+0000 7f3a3dffb700 1 -- 172.21.15.178:0/1366807302 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a40020070 con 0x7f3a4815a5f0 2023-06-28T20:07:04.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.170+0000 7f3a4f820700 1 -- 172.21.15.178:0/1366807302 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a4815b2f0 con 0x7f3a4815a5f0 2023-06-28T20:07:04.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.170+0000 7f3a3dffb700 1 -- 172.21.15.178:0/1366807302 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a40004280 con 0x7f3a4815a5f0 2023-06-28T20:07:04.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.170+0000 7f3a4f820700 1 -- 172.21.15.178:0/1366807302 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a481f3b30 con 0x7f3a4815a5f0 2023-06-28T20:07:04.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.170+0000 7f3a3dffb700 1 -- 172.21.15.178:0/1366807302 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3a400046e0 con 0x7f3a4815a5f0 2023-06-28T20:07:04.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.170+0000 7f3a3dffb700 1 -- 172.21.15.178:0/1366807302 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f3a40022070 con 0x7f3a4815a5f0 2023-06-28T20:07:04.178 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.171+0000 7f3a4f820700 1 -- 172.21.15.178:0/1366807302 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a480728b0 con 0x7f3a4815a5f0 2023-06-28T20:07:04.179 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.173+0000 7f3a3dffb700 1 -- 172.21.15.178:0/1366807302 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f3a4002c060 con 0x7f3a4815a5f0 2023-06-28T20:07:04.262 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.258+0000 7f3a4f820700 1 -- 172.21.15.178:0/1366807302 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b9d323dd-4ed4-4710-8eb6-713385fafe44", "id": 5} v 0) v1 -- 0x7f3a4815dd90 con 0x7f3a4815a5f0 2023-06-28T20:07:04.275 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.271+0000 7f3a3dffb700 1 -- 172.21.15.178:0/1366807302 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b9d323dd-4ed4-4710-8eb6-713385fafe44", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3a40027480 con 0x7f3a4815a5f0 2023-06-28T20:07:04.275 INFO:teuthology.orchestra.run.smithi178.stdout:5 2023-06-28T20:07:04.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.272+0000 7f3a4f820700 1 -- 172.21.15.178:0/1366807302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a4815a5f0 msgr2=0x7f3a481517c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:04.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.272+0000 7f3a4f820700 1 --2- 172.21.15.178:0/1366807302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a4815a5f0 0x7f3a481517c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3a40016000 tx=0x7f3a40004ad0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.276 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.272+0000 7f3a4f820700 1 -- 172.21.15.178:0/1366807302 shutdown_connections 2023-06-28T20:07:04.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.272+0000 7f3a4f820700 1 --2- 172.21.15.178:0/1366807302 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a4815a5f0 0x7f3a481517c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.272+0000 7f3a4f820700 1 --2- 172.21.15.178:0/1366807302 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a480afdf0 0x7f3a481f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.272+0000 7f3a4f820700 1 --2- 172.21.15.178:0/1366807302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a480ceee0 0x7f3a480b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.272+0000 7f3a4f820700 1 -- 172.21.15.178:0/1366807302 >> 172.21.15.178:0/1366807302 conn(0x7f3a480664b0 msgr2=0x7f3a48159620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:04.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.273+0000 7f3a4f820700 1 -- 172.21.15.178:0/1366807302 shutdown_connections 2023-06-28T20:07:04.277 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.273+0000 7f3a4f820700 1 -- 172.21.15.178:0/1366807302 wait complete. 2023-06-28T20:07:04.288 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new e0f5c961-804d-43eb-9668-00b880ddc51b 6 2023-06-28T20:07:04.407 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.403+0000 7f82c88e2700 1 Processor -- start 2023-06-28T20:07:04.408 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.403+0000 7f82c88e2700 1 -- start start 2023-06-28T20:07:04.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.404+0000 7f82c88e2700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f82c415a5f0 0x7f82c40ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:04.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.404+0000 7f82c88e2700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f82c406e320 0x7f82c40ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:04.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.404+0000 7f82c88e2700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82c40ae950 con 0x7f82c40ad7d0 2023-06-28T20:07:04.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.404+0000 7f82c88e2700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c415a9c0 con 0x7f82c415a5f0 2023-06-28T20:07:04.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.404+0000 7f82c88e2700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82c40add40 con 0x7f82c406e320 2023-06-28T20:07:04.410 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.405+0000 7f82c2d9d700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f82c406e320 0x7f82c40ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:04.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.405+0000 7f82c2d9d700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f82c406e320 0x7f82c40ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:47554/0 (socket says 172.21.15.178:47554) 2023-06-28T20:07:04.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.405+0000 7f82c2d9d700 1 -- 172.21.15.178:0/1723496050 learned_addr learned my addr 172.21.15.178:0/1723496050 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-06-28T20:07:04.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.405+0000 7f82c259c700 1 --2- 172.21.15.178:0/1723496050 >> v2:172.21.15.43:3300/0 conn(0x7f82c415a5f0 0x7f82c40ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:04.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.405+0000 7f82c2d9d700 1 -- 172.21.15.178:0/1723496050 >> v2:172.21.15.43:3300/0 conn(0x7f82c415a5f0 msgr2=0x7f82c40ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:04.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.405+0000 7f82c2d9d700 1 --2- 172.21.15.178:0/1723496050 >> v2:172.21.15.43:3300/0 conn(0x7f82c415a5f0 0x7f82c40ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.405+0000 7f82c2d9d700 1 -- 172.21.15.178:0/1723496050 >> v1:172.21.15.43:6789/0 conn(0x7f82c40ad7d0 legacy=0x7f82c40aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:04.411 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.405+0000 7f82c2d9d700 1 -- 172.21.15.178:0/1723496050 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c40ae020 con 0x7f82c406e320 2023-06-28T20:07:04.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.405+0000 7f82c2d9d700 1 --2- 172.21.15.178:0/1723496050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c406e320 0x7f82c40ced10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f82c4071980 tx=0x7f82b400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fcca2082811f4b3 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:04.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.405+0000 7f82c159a700 1 -- 172.21.15.178:0/1723496050 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b400e070 con 0x7f82c406e320 2023-06-28T20:07:04.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.405+0000 7f82c159a700 1 -- 172.21.15.178:0/1723496050 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b4003a20 con 0x7f82c406e320 2023-06-28T20:07:04.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.405+0000 7f82c159a700 1 -- 172.21.15.178:0/1723496050 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b4003dd0 con 0x7f82c406e320 2023-06-28T20:07:04.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.406+0000 7f82c88e2700 1 -- 172.21.15.178:0/1723496050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c406e320 msgr2=0x7f82c40ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:04.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.406+0000 7f82c88e2700 1 --2- 172.21.15.178:0/1723496050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c406e320 0x7f82c40ced10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f82c4071980 tx=0x7f82b400b330 comp rx=0 tx=0).stop 2023-06-28T20:07:04.412 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.406+0000 7f82c88e2700 1 -- 172.21.15.178:0/1723496050 shutdown_connections 2023-06-28T20:07:04.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.406+0000 7f82c88e2700 1 --2- 172.21.15.178:0/1723496050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c406e320 0x7f82c40ced10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.406+0000 7f82c88e2700 1 --2- 172.21.15.178:0/1723496050 >> v2:172.21.15.43:3300/0 conn(0x7f82c415a5f0 0x7f82c40ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.406+0000 7f82c88e2700 1 -- 172.21.15.178:0/1723496050 >> 172.21.15.178:0/1723496050 conn(0x7f82c40664b0 msgr2=0x7f82c40d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:04.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.406+0000 7f82c88e2700 1 -- 172.21.15.178:0/1723496050 shutdown_connections 2023-06-28T20:07:04.413 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.406+0000 7f82c88e2700 1 -- 172.21.15.178:0/1723496050 wait complete. 2023-06-28T20:07:04.414 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:04.407+0000 7f744426c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-28T20:07:04.415 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.407+0000 7f82c88e2700 1 Processor -- start 2023-06-28T20:07:04.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.407+0000 7f82c88e2700 1 -- start start 2023-06-28T20:07:04.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.408+0000 7f82c88e2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c406e320 0x7f82c40b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:04.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.408+0000 7f82c88e2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c40ad7d0 0x7f82c4158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:04.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.408+0000 7f82c88e2700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82c415a5f0 0x7f82c4158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:04.416 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.408+0000 7f82c88e2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c415b0d0 con 0x7f82c406e320 2023-06-28T20:07:04.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.408+0000 7f82c88e2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c40c5450 con 0x7f82c40ad7d0 2023-06-28T20:07:04.417 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.408+0000 7f82c88e2700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c40c4840 con 0x7f82c415a5f0 2023-06-28T20:07:04.418 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.408+0000 7f82c1d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c40ad7d0 0x7f82c4158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:04.418 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.408+0000 7f82c1d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c40ad7d0 0x7f82c4158710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:47558/0 (socket says 172.21.15.178:47558) 2023-06-28T20:07:04.418 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.408+0000 7f82c2d9d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82c415a5f0 0x7f82c4158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:04.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.408+0000 7f82c1d9b700 1 -- 172.21.15.178:0/1301512271 learned_addr learned my addr 172.21.15.178:0/1301512271 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-06-28T20:07:04.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.408+0000 7f82c2d9d700 1 -- 172.21.15.178:0/1301512271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c40ad7d0 msgr2=0x7f82c4158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:04.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.408+0000 7f82c2d9d700 1 --2- 172.21.15.178:0/1301512271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c40ad7d0 0x7f82c4158710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.419 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.408+0000 7f82c2d9d700 1 -- 172.21.15.178:0/1301512271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c406e320 msgr2=0x7f82c40b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:04.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.408+0000 7f82c2d9d700 1 --2- 172.21.15.178:0/1301512271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c406e320 0x7f82c40b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.408+0000 7f82c2d9d700 1 -- 172.21.15.178:0/1301512271 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b4009b30 con 0x7f82c415a5f0 2023-06-28T20:07:04.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.408+0000 7f82c1d9b700 1 --2- 172.21.15.178:0/1301512271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c40ad7d0 0x7f82c4158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:04.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.409+0000 7f82c2d9d700 1 --2- 172.21.15.178:0/1301512271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82c415a5f0 0x7f82c4158c50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f82b4014000 tx=0x7f82b4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:04.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.409+0000 7f82baffd700 1 -- 172.21.15.178:0/1301512271 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b4004440 con 0x7f82c415a5f0 2023-06-28T20:07:04.420 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.409+0000 7f82baffd700 1 -- 172.21.15.178:0/1301512271 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b40045c0 con 0x7f82c415a5f0 2023-06-28T20:07:04.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.409+0000 7f82c88e2700 1 -- 172.21.15.178:0/1301512271 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82c40c4b20 con 0x7f82c415a5f0 2023-06-28T20:07:04.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.409+0000 7f82baffd700 1 -- 172.21.15.178:0/1301512271 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b40094d0 con 0x7f82c415a5f0 2023-06-28T20:07:04.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.409+0000 7f82c88e2700 1 -- 172.21.15.178:0/1301512271 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82c40c50e0 con 0x7f82c415a5f0 2023-06-28T20:07:04.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.409+0000 7f82baffd700 1 -- 172.21.15.178:0/1301512271 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f82b4025750 con 0x7f82c415a5f0 2023-06-28T20:07:04.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.409+0000 7f82baffd700 1 -- 172.21.15.178:0/1301512271 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f82b4012cf0 con 0x7f82c415a5f0 2023-06-28T20:07:04.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.410+0000 7f82c88e2700 1 -- 172.21.15.178:0/1301512271 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82c40c6a60 con 0x7f82c415a5f0 2023-06-28T20:07:04.421 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.412+0000 7f82baffd700 1 -- 172.21.15.178:0/1301512271 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f82b4043b00 con 0x7f82c415a5f0 2023-06-28T20:07:04.499 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.494+0000 7f82c88e2700 1 -- 172.21.15.178:0/1301512271 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e0f5c961-804d-43eb-9668-00b880ddc51b", "id": 6} v 0) v1 -- 0x7f82c40c5270 con 0x7f82c415a5f0 2023-06-28T20:07:04.502 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.498+0000 7f82baffd700 1 -- 172.21.15.178:0/1301512271 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e0f5c961-804d-43eb-9668-00b880ddc51b", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f82b4032070 con 0x7f82c415a5f0 2023-06-28T20:07:04.502 INFO:teuthology.orchestra.run.smithi178.stdout:6 2023-06-28T20:07:04.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.499+0000 7f82c88e2700 1 -- 172.21.15.178:0/1301512271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82c415a5f0 msgr2=0x7f82c4158c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:04.504 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.499+0000 7f82c88e2700 1 --2- 172.21.15.178:0/1301512271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82c415a5f0 0x7f82c4158c50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f82b4014000 tx=0x7f82b4004ad0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.505 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.499+0000 7f82c88e2700 1 -- 172.21.15.178:0/1301512271 shutdown_connections 2023-06-28T20:07:04.505 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.499+0000 7f82c88e2700 1 --2- 172.21.15.178:0/1301512271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f82c415a5f0 0x7f82c4158c50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.505 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.499+0000 7f82c88e2700 1 --2- 172.21.15.178:0/1301512271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c406e320 0x7f82c40b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.505 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.499+0000 7f82c88e2700 1 --2- 172.21.15.178:0/1301512271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82c40ad7d0 0x7f82c4158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.506 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.499+0000 7f82c88e2700 1 -- 172.21.15.178:0/1301512271 >> 172.21.15.178:0/1301512271 conn(0x7f82c40664b0 msgr2=0x7f82c4157fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:04.506 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.499+0000 7f82c88e2700 1 -- 172.21.15.178:0/1301512271 shutdown_connections 2023-06-28T20:07:04.506 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.499+0000 7f82c88e2700 1 -- 172.21.15.178:0/1301512271 wait complete. 2023-06-28T20:07:04.515 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --cluster ceph osd new d6e609c6-b6bc-489b-b844-92552a86b88e 7 2023-06-28T20:07:04.549 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:04.545+0000 7f7d8a535200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-28T20:07:04.572 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:04.568+0000 7f744426c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-28T20:07:04.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.635+0000 7fda5a9b2700 1 Processor -- start 2023-06-28T20:07:04.639 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.635+0000 7fda5a9b2700 1 -- start start 2023-06-28T20:07:04.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.636+0000 7fda5a9b2700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fda5406d310 0x7fda5406cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:04.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.636+0000 7fda5a9b2700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fda5415cae0 0x7fda5406d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:04.641 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.636+0000 7fda5a9b2700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda540d0680 con 0x7fda540cfe00 2023-06-28T20:07:04.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.636+0000 7fda5a9b2700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5406dc20 con 0x7fda5415cae0 2023-06-28T20:07:04.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.636+0000 7fda5a9b2700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5406bb20 con 0x7fda5406d310 2023-06-28T20:07:04.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.636+0000 7fda53fff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fda5406d310 0x7fda5406cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:04.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.636+0000 7fda53fff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fda5406d310 0x7fda5406cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:47562/0 (socket says 172.21.15.178:47562) 2023-06-28T20:07:04.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.636+0000 7fda53fff700 1 -- 172.21.15.178:0/2039233206 learned_addr learned my addr 172.21.15.178:0/2039233206 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-06-28T20:07:04.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.636+0000 7fda537fe700 1 --2- 172.21.15.178:0/2039233206 >> v2:172.21.15.43:3300/0 conn(0x7fda5415cae0 0x7fda5406d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:04.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.636+0000 7fda53fff700 1 -- 172.21.15.178:0/2039233206 >> v2:172.21.15.43:3300/0 conn(0x7fda5415cae0 msgr2=0x7fda5406d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:04.642 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.636+0000 7fda53fff700 1 --2- 172.21.15.178:0/2039233206 >> v2:172.21.15.43:3300/0 conn(0x7fda5415cae0 0x7fda5406d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.636+0000 7fda53fff700 1 -- 172.21.15.178:0/2039233206 >> v1:172.21.15.43:6789/0 conn(0x7fda540cfe00 legacy=0x7fda540ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:04.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.636+0000 7fda53fff700 1 -- 172.21.15.178:0/2039233206 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5406be00 con 0x7fda5406d310 2023-06-28T20:07:04.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.636+0000 7fda537fe700 1 --2- 172.21.15.178:0/2039233206 >> v2:172.21.15.43:3300/0 conn(0x7fda5415cae0 0x7fda5406d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:04.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.636+0000 7fda53fff700 1 --2- 172.21.15.178:0/2039233206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda5406d310 0x7fda5406cbc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fda54066be0 tx=0x7fda44009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aea087b6a0041f server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:04.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.636+0000 7fda52ffd700 1 -- 172.21.15.178:0/2039233206 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda44005e30 con 0x7fda5406d310 2023-06-28T20:07:04.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.637+0000 7fda52ffd700 1 -- 172.21.15.178:0/2039233206 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda44004030 con 0x7fda5406d310 2023-06-28T20:07:04.643 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.637+0000 7fda52ffd700 1 -- 172.21.15.178:0/2039233206 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda440043e0 con 0x7fda5406d310 2023-06-28T20:07:04.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.637+0000 7fda5a9b2700 1 -- 172.21.15.178:0/2039233206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda5406d310 msgr2=0x7fda5406cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:04.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.637+0000 7fda5a9b2700 1 --2- 172.21.15.178:0/2039233206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda5406d310 0x7fda5406cbc0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fda54066be0 tx=0x7fda44009bd0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.637+0000 7fda5a9b2700 1 -- 172.21.15.178:0/2039233206 shutdown_connections 2023-06-28T20:07:04.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.637+0000 7fda5a9b2700 1 --2- 172.21.15.178:0/2039233206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda5406d310 0x7fda5406cbc0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.637+0000 7fda5a9b2700 1 --2- 172.21.15.178:0/2039233206 >> v2:172.21.15.43:3300/0 conn(0x7fda5415cae0 0x7fda5406d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.637+0000 7fda5a9b2700 1 -- 172.21.15.178:0/2039233206 >> 172.21.15.178:0/2039233206 conn(0x7fda540664b0 msgr2=0x7fda540d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:04.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.637+0000 7fda5a9b2700 1 -- 172.21.15.178:0/2039233206 shutdown_connections 2023-06-28T20:07:04.644 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.637+0000 7fda5a9b2700 1 -- 172.21.15.178:0/2039233206 wait complete. 2023-06-28T20:07:04.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.639+0000 7fda5a9b2700 1 Processor -- start 2023-06-28T20:07:04.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.639+0000 7fda5a9b2700 1 -- start start 2023-06-28T20:07:04.646 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.640+0000 7fda5a9b2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda5406d310 0x7fda541e6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:04.647 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.640+0000 7fda5a9b2700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda540cfe00 0x7fda541f1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:04.647 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.640+0000 7fda5a9b2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda5415cae0 0x7fda541e42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:04.647 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.640+0000 7fda5a9b2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda54151950 con 0x7fda5406d310 2023-06-28T20:07:04.647 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.640+0000 7fda5a9b2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda541e6840 con 0x7fda5415cae0 2023-06-28T20:07:04.647 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.640+0000 7fda5a9b2700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda541591b0 con 0x7fda540cfe00 2023-06-28T20:07:04.647 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.640+0000 7fda58f4f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda5415cae0 0x7fda541e42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:04.647 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.640+0000 7fda53fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda5406d310 0x7fda541e6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:04.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.640+0000 7fda58f4f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda5415cae0 0x7fda541e42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:47570/0 (socket says 172.21.15.178:47570) 2023-06-28T20:07:04.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.640+0000 7fda58f4f700 1 -- 172.21.15.178:0/202695282 learned_addr learned my addr 172.21.15.178:0/202695282 (peer_addr_for_me v2:172.21.15.178:0/0) 2023-06-28T20:07:04.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.640+0000 7fda537fe700 1 --2- 172.21.15.178:0/202695282 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda540cfe00 0x7fda541f1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:04.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.640+0000 7fda58f4f700 1 -- 172.21.15.178:0/202695282 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda540cfe00 msgr2=0x7fda541f1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:04.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.640+0000 7fda58f4f700 1 --2- 172.21.15.178:0/202695282 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda540cfe00 0x7fda541f1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.640+0000 7fda58f4f700 1 -- 172.21.15.178:0/202695282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda5406d310 msgr2=0x7fda541e6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:04.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.640+0000 7fda58f4f700 1 --2- 172.21.15.178:0/202695282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda5406d310 0x7fda541e6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.648 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.640+0000 7fda58f4f700 1 -- 172.21.15.178:0/202695282 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda44009500 con 0x7fda5415cae0 2023-06-28T20:07:04.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.640+0000 7fda53fff700 1 --2- 172.21.15.178:0/202695282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda5406d310 0x7fda541e6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:04.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.640+0000 7fda58f4f700 1 --2- 172.21.15.178:0/202695282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda5415cae0 0x7fda541e42c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00cc50 tx=0x7fda4c00cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:04.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.641+0000 7fda50ff9700 1 -- 172.21.15.178:0/202695282 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c003c00 con 0x7fda5415cae0 2023-06-28T20:07:04.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.641+0000 7fda50ff9700 1 -- 172.21.15.178:0/202695282 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4c003d60 con 0x7fda5415cae0 2023-06-28T20:07:04.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.641+0000 7fda5a9b2700 1 -- 172.21.15.178:0/202695282 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda540b0660 con 0x7fda5415cae0 2023-06-28T20:07:04.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.641+0000 7fda50ff9700 1 -- 172.21.15.178:0/202695282 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c00aab0 con 0x7fda5415cae0 2023-06-28T20:07:04.649 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.641+0000 7fda5a9b2700 1 -- 172.21.15.178:0/202695282 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda540b0c20 con 0x7fda5415cae0 2023-06-28T20:07:04.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.641+0000 7fda50ff9700 1 -- 172.21.15.178:0/202695282 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fda4c0050d0 con 0x7fda5415cae0 2023-06-28T20:07:04.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.641+0000 7fda50ff9700 1 -- 172.21.15.178:0/202695282 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fda4c012bc0 con 0x7fda5415cae0 2023-06-28T20:07:04.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.642+0000 7fda5a9b2700 1 -- 172.21.15.178:0/202695282 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda540b0de0 con 0x7fda5415cae0 2023-06-28T20:07:04.650 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.644+0000 7fda50ff9700 1 -- 172.21.15.178:0/202695282 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fda4c012e20 con 0x7fda5415cae0 2023-06-28T20:07:04.731 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.727+0000 7fda5a9b2700 1 -- 172.21.15.178:0/202695282 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d6e609c6-b6bc-489b-b844-92552a86b88e", "id": 7} v 0) v1 -- 0x7fda540728b0 con 0x7fda5415cae0 2023-06-28T20:07:04.734 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.730+0000 7fda50ff9700 1 -- 172.21.15.178:0/202695282 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d6e609c6-b6bc-489b-b844-92552a86b88e", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fda4c005510 con 0x7fda5415cae0 2023-06-28T20:07:04.734 INFO:teuthology.orchestra.run.smithi178.stdout:7 2023-06-28T20:07:04.735 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.731+0000 7fda5a9b2700 1 -- 172.21.15.178:0/202695282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda5415cae0 msgr2=0x7fda541e42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:04.735 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.731+0000 7fda5a9b2700 1 --2- 172.21.15.178:0/202695282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda5415cae0 0x7fda541e42c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00cc50 tx=0x7fda4c00cf60 comp rx=0 tx=0).stop 2023-06-28T20:07:04.735 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.731+0000 7fda5a9b2700 1 -- 172.21.15.178:0/202695282 shutdown_connections 2023-06-28T20:07:04.735 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.731+0000 7fda5a9b2700 1 --2- 172.21.15.178:0/202695282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda5415cae0 0x7fda541e42c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.735 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.731+0000 7fda5a9b2700 1 --2- 172.21.15.178:0/202695282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda5406d310 0x7fda541e6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.736 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.731+0000 7fda5a9b2700 1 --2- 172.21.15.178:0/202695282 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda540cfe00 0x7fda541f1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.736 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.731+0000 7fda5a9b2700 1 -- 172.21.15.178:0/202695282 >> 172.21.15.178:0/202695282 conn(0x7fda540664b0 msgr2=0x7fda5406afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:04.736 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.731+0000 7fda5a9b2700 1 -- 172.21.15.178:0/202695282 shutdown_connections 2023-06-28T20:07:04.736 INFO:teuthology.orchestra.run.smithi178.stderr:2023-06-28T20:07:04.731+0000 7fda5a9b2700 1 -- 172.21.15.178:0/202695282 wait complete. 2023-06-28T20:07:04.747 INFO:tasks.ceph.osd.0:Restarting daemon 2023-06-28T20:07:04.747 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-06-28T20:07:04.749 INFO:tasks.ceph.osd.0:Started 2023-06-28T20:07:04.750 INFO:tasks.ceph.osd.1:Restarting daemon 2023-06-28T20:07:04.750 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-06-28T20:07:04.752 INFO:tasks.ceph.osd.1:Started 2023-06-28T20:07:04.752 INFO:tasks.ceph.osd.2:Restarting daemon 2023-06-28T20:07:04.753 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-06-28T20:07:04.754 INFO:tasks.ceph.osd.2:Started 2023-06-28T20:07:04.755 INFO:tasks.ceph.osd.3:Restarting daemon 2023-06-28T20:07:04.755 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-06-28T20:07:04.757 INFO:tasks.ceph.osd.3:Started 2023-06-28T20:07:04.757 INFO:tasks.ceph.osd.4:Restarting daemon 2023-06-28T20:07:04.757 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-06-28T20:07:04.789 INFO:tasks.ceph.osd.4:Started 2023-06-28T20:07:04.789 INFO:tasks.ceph.osd.5:Restarting daemon 2023-06-28T20:07:04.790 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-06-28T20:07:04.791 INFO:tasks.ceph.osd.5:Started 2023-06-28T20:07:04.792 INFO:tasks.ceph.osd.6:Restarting daemon 2023-06-28T20:07:04.792 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-06-28T20:07:04.794 INFO:tasks.ceph.osd.6:Started 2023-06-28T20:07:04.795 INFO:tasks.ceph.osd.7:Restarting daemon 2023-06-28T20:07:04.795 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-06-28T20:07:04.797 INFO:tasks.ceph.osd.7:Started 2023-06-28T20:07:04.797 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:07:04.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.935+0000 7f4da119f700 1 Processor -- start 2023-06-28T20:07:04.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.935+0000 7f4da119f700 1 -- start start 2023-06-28T20:07:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.936+0000 7f4da119f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4d9c0ce170 0x7f4d9c0c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.936+0000 7f4da119f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d9c15a960 con 0x7f4d9c0ce870 2023-06-28T20:07:04.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.936+0000 7f4da119f700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d9c0ca490 con 0x7f4d9c14ff80 2023-06-28T20:07:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.936+0000 7f4da119f700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d9c0ca770 con 0x7f4d9c0ce170 2023-06-28T20:07:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.936+0000 7f4d9a59c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4d9c0ce170 0x7f4d9c0c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.936+0000 7f4d9b59e700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4d9c0ce870 0x7f4d9c069ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41438/0 (socket says 172.21.15.43:41438) 2023-06-28T20:07:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.936+0000 7f4d9a59c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4d9c0ce170 0x7f4d9c0c9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52250/0 (socket says 172.21.15.43:52250) 2023-06-28T20:07:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.936+0000 7f4d9b59e700 1 -- 172.21.15.43:0/1849744903 learned_addr learned my addr 172.21.15.43:0/1849744903 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.936+0000 7f4d9a59c700 1 -- 172.21.15.43:0/1849744903 >> v1:172.21.15.178:6789/0 conn(0x7f4d9c14ff80 legacy=0x7f4d9c150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:04.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.936+0000 7f4d9a59c700 1 -- 172.21.15.43:0/1849744903 >> v1:172.21.15.43:6789/0 conn(0x7f4d9c0ce870 legacy=0x7f4d9c069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:04.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.936+0000 7f4d9a59c700 1 -- 172.21.15.43:0/1849744903 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9c0cd6b0 con 0x7f4d9c0ce170 2023-06-28T20:07:04.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.936+0000 7f4d99d9b700 1 -- 172.21.15.43:0/1849744903 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4263010901 0 0) 0x7f4d9c15a960 con 0x7f4d9c0ce870 2023-06-28T20:07:04.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.936+0000 7f4d9a59c700 1 --2- 172.21.15.43:0/1849744903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d9c0ce170 0x7f4d9c0c9c20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009840 tx=0x7f4d90009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630d53b3f3b0ba3b server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:04.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.936+0000 7f4d99d9b700 1 -- 172.21.15.43:0/1849744903 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d90011830 con 0x7f4d9c0ce170 2023-06-28T20:07:04.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.936+0000 7f4d99d9b700 1 -- 172.21.15.43:0/1849744903 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d90011990 con 0x7f4d9c0ce170 2023-06-28T20:07:04.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.936+0000 7f4d99d9b700 1 -- 172.21.15.43:0/1849744903 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d90011d20 con 0x7f4d9c0ce170 2023-06-28T20:07:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.937+0000 7f4da119f700 1 -- 172.21.15.43:0/1849744903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d9c0ce170 msgr2=0x7f4d9c0c9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.937+0000 7f4da119f700 1 --2- 172.21.15.43:0/1849744903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d9c0ce170 0x7f4d9c0c9c20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009840 tx=0x7f4d90009b50 comp rx=0 tx=0).stop 2023-06-28T20:07:04.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.937+0000 7f4da119f700 1 -- 172.21.15.43:0/1849744903 shutdown_connections 2023-06-28T20:07:04.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.937+0000 7f4da119f700 1 --2- 172.21.15.43:0/1849744903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d9c0ce170 0x7f4d9c0c9c20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.937+0000 7f4da119f700 1 -- 172.21.15.43:0/1849744903 >> 172.21.15.43:0/1849744903 conn(0x7f4d9c06c170 msgr2=0x7f4d9c06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:04.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.937+0000 7f4da119f700 1 -- 172.21.15.43:0/1849744903 shutdown_connections 2023-06-28T20:07:04.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.937+0000 7f4da119f700 1 -- 172.21.15.43:0/1849744903 wait complete. 2023-06-28T20:07:04.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.939+0000 7f4da119f700 1 Processor -- start 2023-06-28T20:07:04.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.939+0000 7f4da119f700 1 -- start start 2023-06-28T20:07:04.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.940+0000 7f4da119f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d9c0ce170 0x7f4d9c0c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:04.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.940+0000 7f4da119f700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d9c0ce870 0x7f4d9c0b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:04.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.940+0000 7f4da119f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d9c14ff80 0x7f4d9c0b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:04.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.940+0000 7f4da119f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c14c400 con 0x7f4d9c0ce170 2023-06-28T20:07:04.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.940+0000 7f4da119f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c148200 con 0x7f4d9c14ff80 2023-06-28T20:07:04.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.940+0000 7f4da119f700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c0b4a00 con 0x7f4d9c0ce870 2023-06-28T20:07:04.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.940+0000 7f4d9ad9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d9c0ce170 0x7f4d9c0c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:04.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.940+0000 7f4d9ad9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d9c0ce170 0x7f4d9c0c8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52254/0 (socket says 172.21.15.43:52254) 2023-06-28T20:07:04.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.940+0000 7f4d9ad9d700 1 -- 172.21.15.43:0/2406681121 learned_addr learned my addr 172.21.15.43:0/2406681121 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:04.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.940+0000 7f4d9a59c700 1 --2- 172.21.15.43:0/2406681121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d9c0ce870 0x7f4d9c0b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:04.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.940+0000 7f4d9b59e700 1 --2- 172.21.15.43:0/2406681121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d9c14ff80 0x7f4d9c0b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:04.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.940+0000 7f4d9ad9d700 1 -- 172.21.15.43:0/2406681121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d9c0ce870 msgr2=0x7f4d9c0b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:04.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.940+0000 7f4d9ad9d700 1 --2- 172.21.15.43:0/2406681121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d9c0ce870 0x7f4d9c0b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.940+0000 7f4d9ad9d700 1 -- 172.21.15.43:0/2406681121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d9c14ff80 msgr2=0x7f4d9c0b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:04.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.940+0000 7f4d9ad9d700 1 --2- 172.21.15.43:0/2406681121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d9c14ff80 0x7f4d9c0b44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:04.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.940+0000 7f4d9ad9d700 1 -- 172.21.15.43:0/2406681121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d900094a0 con 0x7f4d9c0ce170 2023-06-28T20:07:04.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.941+0000 7f4d9ad9d700 1 --2- 172.21.15.43:0/2406681121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d9c0ce170 0x7f4d9c0c8cd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c00e740 tx=0x7f4d8c00ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:04.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.941+0000 7f4d837fe700 1 -- 172.21.15.43:0/2406681121 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c0052d0 con 0x7f4d9c0ce170 2023-06-28T20:07:04.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.941+0000 7f4d837fe700 1 -- 172.21.15.43:0/2406681121 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8c005430 con 0x7f4d9c0ce170 2023-06-28T20:07:04.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.941+0000 7f4d837fe700 1 -- 172.21.15.43:0/2406681121 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c0108e0 con 0x7f4d9c0ce170 2023-06-28T20:07:04.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.941+0000 7f4da119f700 1 -- 172.21.15.43:0/2406681121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9c15c450 con 0x7f4d9c0ce170 2023-06-28T20:07:04.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.941+0000 7f4da119f700 1 -- 172.21.15.43:0/2406681121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d9c14bb10 con 0x7f4d9c0ce170 2023-06-28T20:07:04.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.941+0000 7f4d837fe700 1 -- 172.21.15.43:0/2406681121 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4d8c010d40 con 0x7f4d9c0ce170 2023-06-28T20:07:04.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.941+0000 7f4d837fe700 1 -- 172.21.15.43:0/2406681121 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f4d8c011b40 con 0x7f4d9c0ce170 2023-06-28T20:07:04.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.942+0000 7f4da119f700 1 -- 172.21.15.43:0/2406681121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d9c0728b0 con 0x7f4d9c0ce170 2023-06-28T20:07:04.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:04.944+0000 7f4d837fe700 1 -- 172.21.15.43:0/2406681121 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f4d8c01f060 con 0x7f4d9c0ce170 2023-06-28T20:07:05.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:05.015+0000 7f4da119f700 1 -- 172.21.15.43:0/2406681121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d9c14c0e0 con 0x7f4d9c0ce170 2023-06-28T20:07:05.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:05.015+0000 7f4d837fe700 1 -- 172.21.15.43:0/2406681121 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f4d8c028090 con 0x7f4d9c0ce170 2023-06-28T20:07:05.019 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:05.019 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":10,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:07:04.729846+0000","last_up_change":"0.000000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:07:05.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:05.016+0000 7f4da119f700 1 -- 172.21.15.43:0/2406681121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d9c0ce170 msgr2=0x7f4d9c0c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:05.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:05.016+0000 7f4da119f700 1 --2- 172.21.15.43:0/2406681121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d9c0ce170 0x7f4d9c0c8cd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c00e740 tx=0x7f4d8c00ea50 comp rx=0 tx=0).stop 2023-06-28T20:07:05.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:05.016+0000 7f4da119f700 1 -- 172.21.15.43:0/2406681121 shutdown_connections 2023-06-28T20:07:05.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:05.016+0000 7f4da119f700 1 --2- 172.21.15.43:0/2406681121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d9c14ff80 0x7f4d9c0b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:05.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:05.016+0000 7f4da119f700 1 --2- 172.21.15.43:0/2406681121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d9c0ce170 0x7f4d9c0c8cd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:05.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:05.016+0000 7f4da119f700 1 --2- 172.21.15.43:0/2406681121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4d9c0ce870 0x7f4d9c0b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:05.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:05.016+0000 7f4da119f700 1 -- 172.21.15.43:0/2406681121 >> 172.21.15.43:0/2406681121 conn(0x7f4d9c06c170 msgr2=0x7f4d9c0cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:05.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:05.016+0000 7f4da119f700 1 -- 172.21.15.43:0/2406681121 shutdown_connections 2023-06-28T20:07:05.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:05.016+0000 7f4da119f700 1 -- 172.21.15.43:0/2406681121 wait complete. 2023-06-28T20:07:05.032 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-06-28T20:07:05.033 INFO:tasks.ceph:Waiting for OSDs to come up 2023-06-28T20:07:05.137 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:05.134+0000 7f7d8a535200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-28T20:07:05.297 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:05.293+0000 7f7d8a535200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-28T20:07:05.344 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:05.340+0000 7f744426c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-28T20:07:05.387 INFO:tasks.ceph.osd.0.smithi043.stderr:2023-06-28T20:07:05.361+0000 7f51fca86640 -1 Falling back to public interface 2023-06-28T20:07:05.387 INFO:tasks.ceph.osd.1.smithi043.stderr:2023-06-28T20:07:05.368+0000 7fc686afb640 -1 Falling back to public interface 2023-06-28T20:07:05.388 INFO:tasks.ceph.osd.3.smithi043.stderr:2023-06-28T20:07:05.374+0000 7f2473854640 -1 Falling back to public interface 2023-06-28T20:07:05.388 INFO:tasks.ceph.osd.2.smithi043.stderr:2023-06-28T20:07:05.378+0000 7fde0c343640 -1 Falling back to public interface 2023-06-28T20:07:05.411 INFO:tasks.ceph.osd.6.smithi178.stderr:2023-06-28T20:07:05.407+0000 7f49d718a640 -1 Falling back to public interface 2023-06-28T20:07:05.413 INFO:tasks.ceph.osd.5.smithi178.stderr:2023-06-28T20:07:05.409+0000 7fc3ca6a7640 -1 Falling back to public interface 2023-06-28T20:07:05.419 INFO:tasks.ceph.osd.7.smithi178.stderr:2023-06-28T20:07:05.415+0000 7fdd97200640 -1 Falling back to public interface 2023-06-28T20:07:05.420 INFO:tasks.ceph.osd.4.smithi178.stderr:2023-06-28T20:07:05.416+0000 7f3a9c658640 -1 Falling back to public interface 2023-06-28T20:07:05.420 INFO:tasks.ceph.osd.4.smithi178.stderr:2023-06-28T20:07:05.416+0000 7f3a93a16700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6819/0: (98) Address already in use 2023-06-28T20:07:05.428 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:05.424+0000 7f744426c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-28T20:07:05.573 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:05.569+0000 7f744426c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-28T20:07:05.823 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:05.819+0000 7f744426c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-28T20:07:05.837 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-06-28T20:07:05.902 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:05.898+0000 7f744426c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-28T20:07:05.956 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.950+0000 7fe045739700 1 Processor -- start 2023-06-28T20:07:05.956 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.950+0000 7fe045739700 1 -- start start 2023-06-28T20:07:05.956 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.951+0000 7fe045739700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe0400cc600 0x7fe04015a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:05.956 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.951+0000 7fe045739700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe040159820 0x7fe0400cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:05.956 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.951+0000 7fe045739700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0400ccf10 con 0x7fe04006ee20 2023-06-28T20:07:05.957 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.951+0000 7fe045739700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0400ce0a0 con 0x7fe040159820 2023-06-28T20:07:05.957 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.951+0000 7fe045739700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0400ce380 con 0x7fe0400cc600 2023-06-28T20:07:05.957 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.951+0000 7fe03e7fc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe040159820 0x7fe0400cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:05.957 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.951+0000 7fe03e7fc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe040159820 0x7fe0400cc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52260/0 (socket says 172.21.15.43:52260) 2023-06-28T20:07:05.957 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.951+0000 7fe03e7fc700 1 -- 172.21.15.43:0/1813113347 learned_addr learned my addr 172.21.15.43:0/1813113347 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:05.957 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.951+0000 7fe03effd700 1 --2- 172.21.15.43:0/1813113347 >> v2:172.21.15.178:3300/0 conn(0x7fe0400cc600 0x7fe04015a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:05.957 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.952+0000 7fe03e7fc700 1 -- 172.21.15.43:0/1813113347 >> v2:172.21.15.178:3300/0 conn(0x7fe0400cc600 msgr2=0x7fe04015a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:05.958 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.952+0000 7fe03e7fc700 1 --2- 172.21.15.43:0/1813113347 >> v2:172.21.15.178:3300/0 conn(0x7fe0400cc600 0x7fe04015a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:05.958 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.952+0000 7fe03e7fc700 1 -- 172.21.15.43:0/1813113347 >> v1:172.21.15.178:6789/0 conn(0x7fe04006ee20 legacy=0x7fe0400cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:05.958 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.952+0000 7fe03e7fc700 1 -- 172.21.15.43:0/1813113347 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe040159c90 con 0x7fe040159820 2023-06-28T20:07:05.958 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.952+0000 7fe03e7fc700 1 --2- 172.21.15.43:0/1813113347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe040159820 0x7fe0400cc9d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe0400720f0 tx=0x7fe034009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b105e0f22dd19eb9 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:05.958 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.952+0000 7fe03dffb700 1 -- 172.21.15.43:0/1813113347 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe034011830 con 0x7fe040159820 2023-06-28T20:07:05.958 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.952+0000 7fe03dffb700 1 -- 172.21.15.43:0/1813113347 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe034011990 con 0x7fe040159820 2023-06-28T20:07:05.958 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.952+0000 7fe03dffb700 1 -- 172.21.15.43:0/1813113347 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe034011d20 con 0x7fe040159820 2023-06-28T20:07:05.961 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.953+0000 7fe045739700 1 -- 172.21.15.43:0/1813113347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe040159820 msgr2=0x7fe0400cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:05.962 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.953+0000 7fe045739700 1 --2- 172.21.15.43:0/1813113347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe040159820 0x7fe0400cc9d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe0400720f0 tx=0x7fe034009bd0 comp rx=0 tx=0).stop 2023-06-28T20:07:05.962 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.953+0000 7fe045739700 1 -- 172.21.15.43:0/1813113347 shutdown_connections 2023-06-28T20:07:05.962 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.953+0000 7fe045739700 1 --2- 172.21.15.43:0/1813113347 >> v2:172.21.15.178:3300/0 conn(0x7fe0400cc600 0x7fe04015a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:05.962 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.953+0000 7fe045739700 1 --2- 172.21.15.43:0/1813113347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe040159820 0x7fe0400cc9d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:05.962 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.953+0000 7fe045739700 1 -- 172.21.15.43:0/1813113347 >> 172.21.15.43:0/1813113347 conn(0x7fe0400b1720 msgr2=0x7fe0400b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:05.962 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.953+0000 7fe045739700 1 -- 172.21.15.43:0/1813113347 shutdown_connections 2023-06-28T20:07:05.962 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.953+0000 7fe045739700 1 -- 172.21.15.43:0/1813113347 wait complete. 2023-06-28T20:07:05.963 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.954+0000 7fe045739700 1 Processor -- start 2023-06-28T20:07:05.963 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.954+0000 7fe045739700 1 -- start start 2023-06-28T20:07:05.963 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.955+0000 7fe045739700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe04006ee20 0x7fe0400c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:05.963 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.955+0000 7fe045739700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0400cc600 0x7fe040149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:05.963 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.955+0000 7fe045739700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe040159820 0x7fe0400c4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:05.963 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.955+0000 7fe045739700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0400c56d0 con 0x7fe0400cc600 2023-06-28T20:07:05.963 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.955+0000 7fe045739700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe040149580 con 0x7fe040159820 2023-06-28T20:07:05.964 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.955+0000 7fe045739700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0400c4ac0 con 0x7fe04006ee20 2023-06-28T20:07:05.964 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.955+0000 7fe03e7fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0400cc600 0x7fe040149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:05.964 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.956+0000 7fe03f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe040159820 0x7fe0400c4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:05.964 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.956+0000 7fe03e7fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0400cc600 0x7fe040149040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52264/0 (socket says 172.21.15.43:52264) 2023-06-28T20:07:05.964 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.956+0000 7fe03e7fc700 1 -- 172.21.15.43:0/2630322663 learned_addr learned my addr 172.21.15.43:0/2630322663 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:05.964 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.956+0000 7fe03effd700 1 --2- 172.21.15.43:0/2630322663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe04006ee20 0x7fe0400c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:05.964 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.956+0000 7fe03e7fc700 1 -- 172.21.15.43:0/2630322663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe04006ee20 msgr2=0x7fe0400c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:05.965 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.956+0000 7fe03e7fc700 1 --2- 172.21.15.43:0/2630322663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe04006ee20 0x7fe0400c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:05.965 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.956+0000 7fe03e7fc700 1 -- 172.21.15.43:0/2630322663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe040159820 msgr2=0x7fe0400c4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:05.965 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.956+0000 7fe03e7fc700 1 --2- 172.21.15.43:0/2630322663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe040159820 0x7fe0400c4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:05.965 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.956+0000 7fe03e7fc700 1 -- 172.21.15.43:0/2630322663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe034009550 con 0x7fe0400cc600 2023-06-28T20:07:05.965 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.956+0000 7fe03f7fe700 1 --2- 172.21.15.43:0/2630322663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe040159820 0x7fe0400c4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:05.965 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.956+0000 7fe03e7fc700 1 --2- 172.21.15.43:0/2630322663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0400cc600 0x7fe040149040 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe034009c10 tx=0x7fe034004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:05.965 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.956+0000 7fe0277fe700 1 -- 172.21.15.43:0/2630322663 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe034004510 con 0x7fe0400cc600 2023-06-28T20:07:05.965 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.956+0000 7fe0277fe700 1 -- 172.21.15.43:0/2630322663 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03401e070 con 0x7fe0400cc600 2023-06-28T20:07:05.966 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.956+0000 7fe045739700 1 -- 172.21.15.43:0/2630322663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0400c4da0 con 0x7fe0400cc600 2023-06-28T20:07:05.966 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.956+0000 7fe0277fe700 1 -- 172.21.15.43:0/2630322663 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0340038e0 con 0x7fe0400cc600 2023-06-28T20:07:05.966 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.956+0000 7fe045739700 1 -- 172.21.15.43:0/2630322663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0400c5360 con 0x7fe0400cc600 2023-06-28T20:07:05.966 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.956+0000 7fe0277fe700 1 -- 172.21.15.43:0/2630322663 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe034003d40 con 0x7fe0400cc600 2023-06-28T20:07:05.966 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.956+0000 7fe0277fe700 1 -- 172.21.15.43:0/2630322663 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fe03402dc20 con 0x7fe0400cc600 2023-06-28T20:07:05.966 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.957+0000 7fe045739700 1 -- 172.21.15.43:0/2630322663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0400728b0 con 0x7fe0400cc600 2023-06-28T20:07:05.967 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:05.959+0000 7fe0277fe700 1 -- 172.21.15.43:0/2630322663 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fe034020070 con 0x7fe0400cc600 2023-06-28T20:07:06.003 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:06.000+0000 7f7d8a535200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-28T20:07:06.036 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:06.032+0000 7fe045739700 1 -- 172.21.15.43:0/2630322663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe04006cbc0 con 0x7fe0400cc600 2023-06-28T20:07:06.036 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:06.032+0000 7fe0277fe700 1 -- 172.21.15.43:0/2630322663 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fe034015020 con 0x7fe0400cc600 2023-06-28T20:07:06.037 INFO:teuthology.misc.health.smithi043.stdout: 2023-06-28T20:07:06.037 INFO:teuthology.misc.health.smithi043.stdout:{"epoch":10,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:07:04.729846+0000","last_up_change":"0.000000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:07:06.038 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:06.033+0000 7fe045739700 1 -- 172.21.15.43:0/2630322663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0400cc600 msgr2=0x7fe040149040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:06.038 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:06.033+0000 7fe045739700 1 --2- 172.21.15.43:0/2630322663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0400cc600 0x7fe040149040 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe034009c10 tx=0x7fe034004750 comp rx=0 tx=0).stop 2023-06-28T20:07:06.038 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:06.033+0000 7fe045739700 1 -- 172.21.15.43:0/2630322663 shutdown_connections 2023-06-28T20:07:06.038 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:06.033+0000 7fe045739700 1 --2- 172.21.15.43:0/2630322663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe040159820 0x7fe0400c4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:06.039 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:06.033+0000 7fe045739700 1 --2- 172.21.15.43:0/2630322663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe04006ee20 0x7fe0400c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:06.039 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:06.033+0000 7fe045739700 1 --2- 172.21.15.43:0/2630322663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0400cc600 0x7fe040149040 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:06.039 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:06.033+0000 7fe045739700 1 -- 172.21.15.43:0/2630322663 >> 172.21.15.43:0/2630322663 conn(0x7fe0400b1720 msgr2=0x7fe04015d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:06.039 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:06.033+0000 7fe045739700 1 -- 172.21.15.43:0/2630322663 shutdown_connections 2023-06-28T20:07:06.039 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:06.033+0000 7fe045739700 1 -- 172.21.15.43:0/2630322663 wait complete. 2023-06-28T20:07:06.049 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-06-28T20:07:06.080 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:06.076+0000 7f7d8a535200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-28T20:07:06.189 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:06.185+0000 7f744426c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-28T20:07:06.219 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:06.215+0000 7f7d8a535200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-28T20:07:06.417 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:06.413+0000 7f744426c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-28T20:07:06.452 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:06.448+0000 7f7d8a535200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-28T20:07:06.526 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:06.523+0000 7f7d8a535200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-28T20:07:06.579 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:06.575+0000 7f744426c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-28T20:07:06.699 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:06.695+0000 7f744426c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-28T20:07:06.801 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:06.798+0000 7f7d8a535200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-28T20:07:06.855 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:06.851+0000 7f744426c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-28T20:07:07.018 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:07.014+0000 7f7d8a535200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-28T20:07:07.141 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:07.137+0000 7f744426c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-28T20:07:07.174 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:07.171+0000 7f7d8a535200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-28T20:07:07.288 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:07.284+0000 7f7d8a535200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-28T20:07:07.333 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:07.330+0000 7f744426c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-28T20:07:07.437 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:07.433+0000 7f7d8a535200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-28T20:07:07.714 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:07.711+0000 7f7d8a535200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-28T20:07:07.899 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:07.895+0000 7f7d8a535200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-28T20:07:07.963 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:07.960+0000 7f744426c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-28T20:07:08.559 INFO:tasks.ceph.mgr.x.smithi178.stderr:2023-06-28T20:07:08.556+0000 7f7d8a535200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-28T20:07:08.882 INFO:tasks.ceph.osd.1.smithi043.stderr:2023-06-28T20:07:08.878+0000 7fc686afb640 -1 osd.1 0 log_to_monitors true 2023-06-28T20:07:08.895 INFO:tasks.ceph.osd.3.smithi043.stderr:2023-06-28T20:07:08.891+0000 7f2473854640 -1 osd.3 0 log_to_monitors true 2023-06-28T20:07:08.898 INFO:tasks.ceph.osd.0.smithi043.stderr:2023-06-28T20:07:08.891+0000 7f51fca86640 -1 osd.0 0 log_to_monitors true 2023-06-28T20:07:08.907 INFO:tasks.ceph.osd.6.smithi178.stderr:2023-06-28T20:07:08.904+0000 7f49d718a640 -1 osd.6 0 log_to_monitors true 2023-06-28T20:07:08.910 INFO:tasks.ceph.osd.2.smithi043.stderr:2023-06-28T20:07:08.907+0000 7fde0c343640 -1 osd.2 0 log_to_monitors true 2023-06-28T20:07:08.971 INFO:tasks.ceph.osd.7.smithi178.stderr:2023-06-28T20:07:08.967+0000 7fdd97200640 -1 osd.7 0 log_to_monitors true 2023-06-28T20:07:08.988 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:08.984+0000 7f73fc4d4700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-06-28T20:07:08.990 INFO:tasks.ceph.osd.5.smithi178.stderr:2023-06-28T20:07:08.987+0000 7fc3ca6a7640 -1 osd.5 0 log_to_monitors true 2023-06-28T20:07:08.991 INFO:tasks.ceph.mgr.y.smithi043.stderr:2023-06-28T20:07:08.987+0000 7f73fc4d4700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-06-28T20:07:08.995 INFO:tasks.ceph.osd.4.smithi178.stderr:2023-06-28T20:07:08.991+0000 7f3a9c658640 -1 osd.4 0 log_to_monitors true 2023-06-28T20:07:11.262 INFO:tasks.ceph.osd.6.smithi178.stderr:2023-06-28T20:07:11.258+0000 7f49cad41700 -1 osd.6 0 waiting for initial osdmap 2023-06-28T20:07:11.279 INFO:tasks.ceph.osd.6.smithi178.stderr:2023-06-28T20:07:11.276+0000 7f49c311c700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-28T20:07:11.280 INFO:tasks.ceph.osd.7.smithi178.stderr:2023-06-28T20:07:11.276+0000 7fdd8adb7700 -1 osd.7 0 waiting for initial osdmap 2023-06-28T20:07:11.291 INFO:tasks.ceph.osd.7.smithi178.stderr:2023-06-28T20:07:11.288+0000 7fdd83192700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-28T20:07:11.319 INFO:tasks.ceph.osd.5.smithi178.stderr:2023-06-28T20:07:11.315+0000 7fc3be25e700 -1 osd.5 0 waiting for initial osdmap 2023-06-28T20:07:11.328 INFO:tasks.ceph.osd.5.smithi178.stderr:2023-06-28T20:07:11.325+0000 7fc3b6639700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-28T20:07:11.330 INFO:tasks.ceph.osd.3.smithi043.stderr:2023-06-28T20:07:11.327+0000 7f246740b700 -1 osd.3 0 waiting for initial osdmap 2023-06-28T20:07:11.344 INFO:tasks.ceph.osd.3.smithi043.stderr:2023-06-28T20:07:11.340+0000 7f245f7e6700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-28T20:07:11.346 INFO:tasks.ceph.osd.2.smithi043.stderr:2023-06-28T20:07:11.342+0000 7fddffefa700 -1 osd.2 0 waiting for initial osdmap 2023-06-28T20:07:11.359 INFO:tasks.ceph.osd.2.smithi043.stderr:2023-06-28T20:07:11.355+0000 7fddf82d5700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-28T20:07:11.372 INFO:tasks.ceph.osd.4.smithi178.stderr:2023-06-28T20:07:11.368+0000 7f3a9020f700 -1 osd.4 0 waiting for initial osdmap 2023-06-28T20:07:11.390 INFO:tasks.ceph.osd.1.smithi043.stderr:2023-06-28T20:07:11.386+0000 7fc67a6b2700 -1 osd.1 0 waiting for initial osdmap 2023-06-28T20:07:11.394 INFO:tasks.ceph.osd.4.smithi178.stderr:2023-06-28T20:07:11.391+0000 7f3a885ea700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-28T20:07:11.409 INFO:tasks.ceph.osd.1.smithi043.stderr:2023-06-28T20:07:11.405+0000 7fc672a8d700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-28T20:07:11.424 INFO:tasks.ceph.osd.0.smithi043.stderr:2023-06-28T20:07:11.420+0000 7f51f063d700 -1 osd.0 0 waiting for initial osdmap 2023-06-28T20:07:11.443 INFO:tasks.ceph.osd.0.smithi043.stderr:2023-06-28T20:07:11.440+0000 7f51e8a18700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-28T20:07:12.857 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-06-28T20:07:12.987 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.983+0000 7f953e0de700 1 Processor -- start 2023-06-28T20:07:12.987 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.983+0000 7f953e0de700 1 -- start start 2023-06-28T20:07:12.989 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.984+0000 7f953e0de700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95380cc600 0x7f953815a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:12.990 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.984+0000 7f953e0de700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9538159820 0x7f95380cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:12.990 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.984+0000 7f953e0de700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f953806ee20 0x7f95380cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:12.990 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.984+0000 7f953e0de700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95380ccf10 con 0x7f9538159820 2023-06-28T20:07:12.990 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.984+0000 7f953e0de700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95380ce0a0 con 0x7f953806ee20 2023-06-28T20:07:12.990 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.984+0000 7f953e0de700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95380ce380 con 0x7f95380cc600 2023-06-28T20:07:12.990 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.984+0000 7f9536ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9538159820 0x7f95380cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:12.991 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.984+0000 7f9536ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9538159820 0x7f95380cc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52502/0 (socket says 172.21.15.43:52502) 2023-06-28T20:07:12.991 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.984+0000 7f9536ffd700 1 -- 172.21.15.43:0/126683520 learned_addr learned my addr 172.21.15.43:0/126683520 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:12.991 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.984+0000 7f95377fe700 1 --2- 172.21.15.43:0/126683520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95380cc600 0x7f953815a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:12.991 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.984+0000 7f9537fff700 1 --2- 172.21.15.43:0/126683520 >> v2:172.21.15.178:3300/0 conn(0x7f953806ee20 0x7f95380cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:12.991 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.984+0000 7f9536ffd700 1 -- 172.21.15.43:0/126683520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95380cc600 msgr2=0x7f953815a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:12.991 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.984+0000 7f9536ffd700 1 --2- 172.21.15.43:0/126683520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95380cc600 0x7f953815a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:12.991 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.984+0000 7f9536ffd700 1 -- 172.21.15.43:0/126683520 >> v2:172.21.15.178:3300/0 conn(0x7f953806ee20 msgr2=0x7f95380cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:12.992 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.984+0000 7f9536ffd700 1 --2- 172.21.15.43:0/126683520 >> v2:172.21.15.178:3300/0 conn(0x7f953806ee20 0x7f95380cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:12.992 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.984+0000 7f9536ffd700 1 -- 172.21.15.43:0/126683520 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9538159c90 con 0x7f9538159820 2023-06-28T20:07:12.992 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.985+0000 7f9536ffd700 1 --2- 172.21.15.43:0/126683520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9538159820 0x7f95380cc9d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f95380cc570 tx=0x7f952c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2eff97d2646e57 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:12.992 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.985+0000 7f95367fc700 1 -- 172.21.15.43:0/126683520 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c00b870 con 0x7f9538159820 2023-06-28T20:07:12.992 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.985+0000 7f95367fc700 1 -- 172.21.15.43:0/126683520 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952c00b9d0 con 0x7f9538159820 2023-06-28T20:07:12.992 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.985+0000 7f95367fc700 1 -- 172.21.15.43:0/126683520 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c00bd60 con 0x7f9538159820 2023-06-28T20:07:12.993 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.985+0000 7f953e0de700 1 -- 172.21.15.43:0/126683520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9538159820 msgr2=0x7f95380cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:12.993 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.985+0000 7f953e0de700 1 --2- 172.21.15.43:0/126683520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9538159820 0x7f95380cc9d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f95380cc570 tx=0x7f952c009bd0 comp rx=0 tx=0).stop 2023-06-28T20:07:12.993 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.985+0000 7f953e0de700 1 -- 172.21.15.43:0/126683520 shutdown_connections 2023-06-28T20:07:12.993 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.985+0000 7f953e0de700 1 --2- 172.21.15.43:0/126683520 >> v2:172.21.15.178:3300/0 conn(0x7f953806ee20 0x7f95380cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:12.994 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.985+0000 7f953e0de700 1 --2- 172.21.15.43:0/126683520 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95380cc600 0x7f953815a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:12.994 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.985+0000 7f953e0de700 1 --2- 172.21.15.43:0/126683520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9538159820 0x7f95380cc9d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:12.994 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.985+0000 7f953e0de700 1 -- 172.21.15.43:0/126683520 >> 172.21.15.43:0/126683520 conn(0x7f95380b1720 msgr2=0x7f95380b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:12.994 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.985+0000 7f953e0de700 1 -- 172.21.15.43:0/126683520 shutdown_connections 2023-06-28T20:07:12.994 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.985+0000 7f953e0de700 1 -- 172.21.15.43:0/126683520 wait complete. 2023-06-28T20:07:12.998 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.987+0000 7f953e0de700 1 Processor -- start 2023-06-28T20:07:12.998 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.987+0000 7f953e0de700 1 -- start start 2023-06-28T20:07:12.998 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.988+0000 7f953e0de700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f953806ee20 0x7f95381f00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:12.999 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.988+0000 7f953e0de700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95380cc600 0x7f95381f11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:12.999 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.988+0000 7f953e0de700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9538159820 0x7f95381f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:12.999 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.988+0000 7f953e0de700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9538158e10 con 0x7f95380cc600 2023-06-28T20:07:12.999 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.988+0000 7f953e0de700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9538157310 con 0x7f953806ee20 2023-06-28T20:07:13.000 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.988+0000 7f953e0de700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9538157570 con 0x7f9538159820 2023-06-28T20:07:13.000 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.988+0000 7f9536ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95380cc600 0x7f95381f11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:13.000 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.988+0000 7f9536ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95380cc600 0x7f95381f11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52510/0 (socket says 172.21.15.43:52510) 2023-06-28T20:07:13.001 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.988+0000 7f9536ffd700 1 -- 172.21.15.43:0/409730461 learned_addr learned my addr 172.21.15.43:0/409730461 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:13.001 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.988+0000 7f95377fe700 1 --2- 172.21.15.43:0/409730461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f953806ee20 0x7f95381f00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:13.001 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.988+0000 7f9537fff700 1 --2- 172.21.15.43:0/409730461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9538159820 0x7f95381f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:13.001 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.988+0000 7f9536ffd700 1 -- 172.21.15.43:0/409730461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9538159820 msgr2=0x7f95381f36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:13.001 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.988+0000 7f9536ffd700 1 --2- 172.21.15.43:0/409730461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9538159820 0x7f95381f36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:13.001 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.988+0000 7f9536ffd700 1 -- 172.21.15.43:0/409730461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f953806ee20 msgr2=0x7f95381f00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:13.002 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.988+0000 7f9536ffd700 1 --2- 172.21.15.43:0/409730461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f953806ee20 0x7f95381f00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:13.002 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.988+0000 7f9536ffd700 1 -- 172.21.15.43:0/409730461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c009550 con 0x7f95380cc600 2023-06-28T20:07:13.002 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.989+0000 7f9536ffd700 1 --2- 172.21.15.43:0/409730461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95380cc600 0x7f95381f11f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f952c000c00 tx=0x7f952c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:13.002 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.989+0000 7f951ffff700 1 -- 172.21.15.43:0/409730461 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c004540 con 0x7f95380cc600 2023-06-28T20:07:13.002 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.989+0000 7f951ffff700 1 -- 172.21.15.43:0/409730461 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952c0036a0 con 0x7f95380cc600 2023-06-28T20:07:13.002 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.989+0000 7f951ffff700 1 -- 172.21.15.43:0/409730461 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952c003a50 con 0x7f95380cc600 2023-06-28T20:07:13.002 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.989+0000 7f953e0de700 1 -- 172.21.15.43:0/409730461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95381f3dd0 con 0x7f95380cc600 2023-06-28T20:07:13.003 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.989+0000 7f953e0de700 1 -- 172.21.15.43:0/409730461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95381f4390 con 0x7f95380cc600 2023-06-28T20:07:13.003 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.990+0000 7f953e0de700 1 -- 172.21.15.43:0/409730461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9538157790 con 0x7f95380cc600 2023-06-28T20:07:13.003 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.990+0000 7f951ffff700 1 -- 172.21.15.43:0/409730461 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f952c021070 con 0x7f95380cc600 2023-06-28T20:07:13.003 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.991+0000 7f951ffff700 1 --2- 172.21.15.43:0/409730461 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f952406ac50 0x7f952406d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:13.003 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.991+0000 7f951ffff700 1 -- 172.21.15.43:0/409730461 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f952c08a3f0 con 0x7f95380cc600 2023-06-28T20:07:13.003 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.991+0000 7f95377fe700 1 --2- 172.21.15.43:0/409730461 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f952406ac50 0x7f952406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:13.003 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.991+0000 7f95377fe700 1 --2- 172.21.15.43:0/409730461 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f952406ac50 0x7f952406d100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9528009930 tx=0x7f9528008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:13.004 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:12.995+0000 7f951ffff700 1 -- 172.21.15.43:0/409730461 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9538157790 con 0x7f95380cc600 2023-06-28T20:07:13.201 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:13.197+0000 7f953e0de700 1 -- 172.21.15.43:0/409730461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9538157790 con 0x7f95380cc600 2023-06-28T20:07:13.202 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:13.198+0000 7f951ffff700 1 -- 172.21.15.43:0/409730461 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10908 (secure 0 0 0) 0x7f9538157790 con 0x7f95380cc600 2023-06-28T20:07:13.202 INFO:teuthology.misc.health.smithi043.stdout: 2023-06-28T20:07:13.203 INFO:teuthology.misc.health.smithi043.stdout:{"epoch":13,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:07:12.279568+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:07:13.204 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:13.200+0000 7f953e0de700 1 -- 172.21.15.43:0/409730461 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f952406ac50 msgr2=0x7f952406d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:13.204 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:13.200+0000 7f953e0de700 1 --2- 172.21.15.43:0/409730461 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f952406ac50 0x7f952406d100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9528009930 tx=0x7f9528008040 comp rx=0 tx=0).stop 2023-06-28T20:07:13.204 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:13.200+0000 7f953e0de700 1 -- 172.21.15.43:0/409730461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95380cc600 msgr2=0x7f95381f11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:13.205 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:13.200+0000 7f953e0de700 1 --2- 172.21.15.43:0/409730461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95380cc600 0x7f95381f11f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f952c000c00 tx=0x7f952c004750 comp rx=0 tx=0).stop 2023-06-28T20:07:13.205 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:13.200+0000 7f953e0de700 1 -- 172.21.15.43:0/409730461 shutdown_connections 2023-06-28T20:07:13.205 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:13.200+0000 7f953e0de700 1 --2- 172.21.15.43:0/409730461 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f952406ac50 0x7f952406d100 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:13.205 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:13.200+0000 7f953e0de700 1 --2- 172.21.15.43:0/409730461 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9538159820 0x7f95381f36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:13.205 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:13.200+0000 7f953e0de700 1 --2- 172.21.15.43:0/409730461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f953806ee20 0x7f95381f00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:13.205 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:13.200+0000 7f953e0de700 1 --2- 172.21.15.43:0/409730461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95380cc600 0x7f95381f11f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:13.205 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:13.200+0000 7f953e0de700 1 -- 172.21.15.43:0/409730461 >> 172.21.15.43:0/409730461 conn(0x7f95380b1720 msgr2=0x7f953806e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:13.206 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:13.200+0000 7f953e0de700 1 -- 172.21.15.43:0/409730461 shutdown_connections 2023-06-28T20:07:13.206 INFO:teuthology.misc.health.smithi043.stderr:2023-06-28T20:07:13.200+0000 7f953e0de700 1 -- 172.21.15.43:0/409730461 wait complete. 2023-06-28T20:07:13.215 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-06-28T20:07:13.216 INFO:tasks.ceph:Creating RBD pool 2023-06-28T20:07:13.216 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd pool create rbd 8 2023-06-28T20:07:13.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.337+0000 7f5f4cb3b700 1 Processor -- start 2023-06-28T20:07:13.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.337+0000 7f5f4cb3b700 1 -- start start 2023-06-28T20:07:13.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.338+0000 7f5f4cb3b700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5f4815a5f0 0x7f5f480ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:13.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.338+0000 7f5f4cb3b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5f480ad7d0 0x7f5f480ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:13.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.338+0000 7f5f4cb3b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f480aede0 0x7f5f480af260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:13.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.338+0000 7f5f4cb3b700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f480ae950 con 0x7f5f480ad7d0 2023-06-28T20:07:13.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.338+0000 7f5f4cb3b700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4815a9c0 con 0x7f5f4815a5f0 2023-06-28T20:07:13.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.338+0000 7f5f4cb3b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f480add40 con 0x7f5f480aede0 2023-06-28T20:07:13.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.338+0000 7f5f45d9b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5f480ad7d0 0x7f5f480ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:13.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.338+0000 7f5f45d9b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5f480ad7d0 0x7f5f480ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52524/0 (socket says 172.21.15.43:52524) 2023-06-28T20:07:13.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.338+0000 7f5f45d9b700 1 -- 172.21.15.43:0/2858438614 learned_addr learned my addr 172.21.15.43:0/2858438614 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:13.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.338+0000 7f5f46d9d700 1 --2- 172.21.15.43:0/2858438614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f480aede0 0x7f5f480af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:13.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.338+0000 7f5f4659c700 1 --2- 172.21.15.43:0/2858438614 >> v2:172.21.15.178:3300/0 conn(0x7f5f4815a5f0 0x7f5f480ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:13.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.338+0000 7f5f45d9b700 1 -- 172.21.15.43:0/2858438614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f480aede0 msgr2=0x7f5f480af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:13.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.338+0000 7f5f45d9b700 1 --2- 172.21.15.43:0/2858438614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f480aede0 0x7f5f480af260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:13.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.338+0000 7f5f45d9b700 1 -- 172.21.15.43:0/2858438614 >> v2:172.21.15.178:3300/0 conn(0x7f5f4815a5f0 msgr2=0x7f5f480ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:13.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.338+0000 7f5f45d9b700 1 --2- 172.21.15.43:0/2858438614 >> v2:172.21.15.178:3300/0 conn(0x7f5f4815a5f0 0x7f5f480ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:13.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.338+0000 7f5f45d9b700 1 -- 172.21.15.43:0/2858438614 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f480ae020 con 0x7f5f480ad7d0 2023-06-28T20:07:13.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.338+0000 7f5f4659c700 1 --2- 172.21.15.43:0/2858438614 >> v2:172.21.15.178:3300/0 conn(0x7f5f4815a5f0 0x7f5f480ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:13.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.338+0000 7f5f45d9b700 1 --2- 172.21.15.43:0/2858438614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f480ad7d0 0x7f5f480ced10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5f48067eb0 tx=0x7f5f3c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f756587c2e411955 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:13.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.338+0000 7f5f4559a700 1 -- 172.21.15.43:0/2858438614 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c011830 con 0x7f5f480ad7d0 2023-06-28T20:07:13.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.338+0000 7f5f4559a700 1 -- 172.21.15.43:0/2858438614 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3c011990 con 0x7f5f480ad7d0 2023-06-28T20:07:13.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.338+0000 7f5f4559a700 1 -- 172.21.15.43:0/2858438614 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c011d20 con 0x7f5f480ad7d0 2023-06-28T20:07:13.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.339+0000 7f5f4cb3b700 1 -- 172.21.15.43:0/2858438614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f480ad7d0 msgr2=0x7f5f480ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:13.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.339+0000 7f5f4cb3b700 1 --2- 172.21.15.43:0/2858438614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f480ad7d0 0x7f5f480ced10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5f48067eb0 tx=0x7f5f3c009bd0 comp rx=0 tx=0).stop 2023-06-28T20:07:13.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.339+0000 7f5f4cb3b700 1 -- 172.21.15.43:0/2858438614 shutdown_connections 2023-06-28T20:07:13.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.339+0000 7f5f4cb3b700 1 --2- 172.21.15.43:0/2858438614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f480aede0 0x7f5f480af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:13.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.339+0000 7f5f4cb3b700 1 --2- 172.21.15.43:0/2858438614 >> v2:172.21.15.178:3300/0 conn(0x7f5f4815a5f0 0x7f5f480ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:13.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.339+0000 7f5f4cb3b700 1 --2- 172.21.15.43:0/2858438614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f480ad7d0 0x7f5f480ced10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:13.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.339+0000 7f5f4cb3b700 1 -- 172.21.15.43:0/2858438614 >> 172.21.15.43:0/2858438614 conn(0x7f5f480664b0 msgr2=0x7f5f480d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:13.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.339+0000 7f5f4cb3b700 1 -- 172.21.15.43:0/2858438614 shutdown_connections 2023-06-28T20:07:13.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.339+0000 7f5f4cb3b700 1 -- 172.21.15.43:0/2858438614 wait complete. 2023-06-28T20:07:13.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.340+0000 7f5f4cb3b700 1 Processor -- start 2023-06-28T20:07:13.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.340+0000 7f5f4cb3b700 1 -- start start 2023-06-28T20:07:13.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.341+0000 7f5f4cb3b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f480ad7d0 0x7f5f48157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:13.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.341+0000 7f5f4cb3b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f480aede0 0x7f5f481467a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:13.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.341+0000 7f5f4cb3b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f4815a5f0 0x7f5f48146ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:13.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.341+0000 7f5f4cb3b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f480b4950 con 0x7f5f480aede0 2023-06-28T20:07:13.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.341+0000 7f5f4cb3b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f48157cc0 con 0x7f5f480ad7d0 2023-06-28T20:07:13.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.341+0000 7f5f4cb3b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f48147590 con 0x7f5f4815a5f0 2023-06-28T20:07:13.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.341+0000 7f5f45d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f480aede0 0x7f5f481467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:13.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.341+0000 7f5f45d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f480aede0 0x7f5f481467a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52532/0 (socket says 172.21.15.43:52532) 2023-06-28T20:07:13.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.341+0000 7f5f45d9b700 1 -- 172.21.15.43:0/2995704418 learned_addr learned my addr 172.21.15.43:0/2995704418 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:13.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.341+0000 7f5f4659c700 1 --2- 172.21.15.43:0/2995704418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f480ad7d0 0x7f5f48157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:13.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.341+0000 7f5f46d9d700 1 --2- 172.21.15.43:0/2995704418 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f4815a5f0 0x7f5f48146ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:13.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.342+0000 7f5f45d9b700 1 -- 172.21.15.43:0/2995704418 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f4815a5f0 msgr2=0x7f5f48146ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:13.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.342+0000 7f5f45d9b700 1 --2- 172.21.15.43:0/2995704418 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f4815a5f0 0x7f5f48146ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:13.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.342+0000 7f5f45d9b700 1 -- 172.21.15.43:0/2995704418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f480ad7d0 msgr2=0x7f5f48157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:13.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.342+0000 7f5f45d9b700 1 --2- 172.21.15.43:0/2995704418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f480ad7d0 0x7f5f48157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:13.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.342+0000 7f5f45d9b700 1 -- 172.21.15.43:0/2995704418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c009550 con 0x7f5f480aede0 2023-06-28T20:07:13.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.342+0000 7f5f4659c700 1 --2- 172.21.15.43:0/2995704418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f480ad7d0 0x7f5f48157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:13.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.342+0000 7f5f46d9d700 1 --2- 172.21.15.43:0/2995704418 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f4815a5f0 0x7f5f48146ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:13.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.342+0000 7f5f45d9b700 1 --2- 172.21.15.43:0/2995704418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f480aede0 0x7f5f481467a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c000c00 tx=0x7f5f3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:13.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.342+0000 7f5f36ffd700 1 -- 172.21.15.43:0/2995704418 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c004300 con 0x7f5f480aede0 2023-06-28T20:07:13.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.342+0000 7f5f36ffd700 1 -- 172.21.15.43:0/2995704418 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3c01f070 con 0x7f5f480aede0 2023-06-28T20:07:13.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.342+0000 7f5f4cb3b700 1 -- 172.21.15.43:0/2995704418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f48147870 con 0x7f5f480aede0 2023-06-28T20:07:13.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.342+0000 7f5f36ffd700 1 -- 172.21.15.43:0/2995704418 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c0036a0 con 0x7f5f480aede0 2023-06-28T20:07:13.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.342+0000 7f5f4cb3b700 1 -- 172.21.15.43:0/2995704418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f48147220 con 0x7f5f480aede0 2023-06-28T20:07:13.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.343+0000 7f5f4cb3b700 1 -- 172.21.15.43:0/2995704418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f480b4440 con 0x7f5f480aede0 2023-06-28T20:07:13.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.343+0000 7f5f36ffd700 1 -- 172.21.15.43:0/2995704418 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f5f3c021030 con 0x7f5f480aede0 2023-06-28T20:07:13.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.344+0000 7f5f36ffd700 1 --2- 172.21.15.43:0/2995704418 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5f2806ac50 0x7f5f2806d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:13.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.344+0000 7f5f36ffd700 1 -- 172.21.15.43:0/2995704418 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f5f3c08a1d0 con 0x7f5f480aede0 2023-06-28T20:07:13.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.344+0000 7f5f4659c700 1 --2- 172.21.15.43:0/2995704418 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5f2806ac50 0x7f5f2806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:13.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.344+0000 7f5f4659c700 1 --2- 172.21.15.43:0/2995704418 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5f2806ac50 0x7f5f2806d100 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5f480b2e40 tx=0x7f5f38008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:13.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.348+0000 7f5f36ffd700 1 -- 172.21.15.43:0/2995704418 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5f480b4440 con 0x7f5f480aede0 2023-06-28T20:07:13.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:13.549+0000 7f5f4cb3b700 1 -- 172.21.15.43:0/2995704418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f5f480b4440 con 0x7f5f480aede0 2023-06-28T20:07:14.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.292+0000 7f5f36ffd700 1 -- 172.21.15.43:0/2995704418 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v15) v1 ==== 109+0+0 (secure 0 0 0) 0x7f5f480b4440 con 0x7f5f480aede0 2023-06-28T20:07:14.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.296+0000 7f5f4cb3b700 1 -- 172.21.15.43:0/2995704418 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5f2806ac50 msgr2=0x7f5f2806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:14.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.296+0000 7f5f4cb3b700 1 --2- 172.21.15.43:0/2995704418 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5f2806ac50 0x7f5f2806d100 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5f480b2e40 tx=0x7f5f38008040 comp rx=0 tx=0).stop 2023-06-28T20:07:14.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.296+0000 7f5f4cb3b700 1 -- 172.21.15.43:0/2995704418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f480aede0 msgr2=0x7f5f481467a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:14.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.296+0000 7f5f4cb3b700 1 --2- 172.21.15.43:0/2995704418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f480aede0 0x7f5f481467a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c000c00 tx=0x7f5f3c004750 comp rx=0 tx=0).stop 2023-06-28T20:07:14.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.296+0000 7f5f4cb3b700 1 -- 172.21.15.43:0/2995704418 shutdown_connections 2023-06-28T20:07:14.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.296+0000 7f5f4cb3b700 1 --2- 172.21.15.43:0/2995704418 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5f2806ac50 0x7f5f2806d100 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:14.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.296+0000 7f5f4cb3b700 1 --2- 172.21.15.43:0/2995704418 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f4815a5f0 0x7f5f48146ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:14.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.296+0000 7f5f4cb3b700 1 --2- 172.21.15.43:0/2995704418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f480ad7d0 0x7f5f48157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:14.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.296+0000 7f5f4cb3b700 1 --2- 172.21.15.43:0/2995704418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f480aede0 0x7f5f481467a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:14.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.296+0000 7f5f4cb3b700 1 -- 172.21.15.43:0/2995704418 >> 172.21.15.43:0/2995704418 conn(0x7f5f480664b0 msgr2=0x7f5f4806c780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:14.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.296+0000 7f5f4cb3b700 1 -- 172.21.15.43:0/2995704418 shutdown_connections 2023-06-28T20:07:14.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.296+0000 7f5f4cb3b700 1 -- 172.21.15.43:0/2995704418 wait complete. 2023-06-28T20:07:14.302 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'rbd' created 2023-06-28T20:07:14.315 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-06-28T20:07:14.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.444+0000 7efc78082700 1 Processor -- start 2023-06-28T20:07:14.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.444+0000 7efc78082700 1 -- start start 2023-06-28T20:07:14.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.445+0000 7efc78082700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc7015d930 0x7efc7006a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:14.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.445+0000 7efc78082700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc70160330 con 0x7efc700aed40 2023-06-28T20:07:14.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.445+0000 7efc78082700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc70160610 con 0x7efc7006e190 2023-06-28T20:07:14.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.445+0000 7efc78082700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc70160fb0 con 0x7efc7015d930 2023-06-28T20:07:14.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.445+0000 7efc7561d700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7efc700aed40 0x7efc7006da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41742/0 (socket says 172.21.15.43:41742) 2023-06-28T20:07:14.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.445+0000 7efc7561d700 1 -- 172.21.15.43:0/3675423085 learned_addr learned my addr 172.21.15.43:0/3675423085 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:14.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc75e1e700 1 --2- 172.21.15.43:0/3675423085 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc7015d930 0x7efc7006a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:14.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc74e1c700 1 -- 172.21.15.43:0/3675423085 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2247961778 0 0) 0x7efc70160330 con 0x7efc700aed40 2023-06-28T20:07:14.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc74e1c700 1 -- 172.21.15.43:0/3675423085 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc5c0035b0 con 0x7efc700aed40 2023-06-28T20:07:14.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc74e1c700 1 -- 172.21.15.43:0/3675423085 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810454566 0 0) 0x7efc70160610 con 0x7efc7006e190 2023-06-28T20:07:14.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc74e1c700 1 -- 172.21.15.43:0/3675423085 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc70160330 con 0x7efc7006e190 2023-06-28T20:07:14.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc74e1c700 1 -- 172.21.15.43:0/3675423085 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1774117260 0 0) 0x7efc5c0035b0 con 0x7efc700aed40 2023-06-28T20:07:14.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc74e1c700 1 -- 172.21.15.43:0/3675423085 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc7015d930 msgr2=0x7efc7006a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:14.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc74e1c700 1 --2- 172.21.15.43:0/3675423085 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc7015d930 0x7efc7006a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:14.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc74e1c700 1 -- 172.21.15.43:0/3675423085 >> v1:172.21.15.178:6789/0 conn(0x7efc7006e190 legacy=0x7efc7006ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:14.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc74e1c700 1 -- 172.21.15.43:0/3675423085 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc701608f0 con 0x7efc700aed40 2023-06-28T20:07:14.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc74e1c700 1 -- 172.21.15.43:0/3675423085 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7efc6c001670 con 0x7efc700aed40 2023-06-28T20:07:14.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc75e1e700 1 --2- 172.21.15.43:0/3675423085 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc7015d930 0x7efc7006a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:14.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc74e1c700 1 -- 172.21.15.43:0/3675423085 >> v1:172.21.15.43:6789/0 conn(0x7efc700aed40 legacy=0x7efc7006da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:14.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc74e1c700 1 --2- 172.21.15.43:0/3675423085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc5c003bc0 0x7efc5c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:14.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc74e1c700 1 --2- 172.21.15.43:0/3675423085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc5c004850 0x7efc5c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:14.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc74e1c700 1 --2- 172.21.15.43:0/3675423085 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc7015d930 0x7efc5c005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:14.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc74e1c700 1 -- 172.21.15.43:0/3675423085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc70160fb0 con 0x7efc5c004850 2023-06-28T20:07:14.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc74e1c700 1 -- 172.21.15.43:0/3675423085 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c00bba0 con 0x7efc5c003bc0 2023-06-28T20:07:14.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc74e1c700 1 -- 172.21.15.43:0/3675423085 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c00bd30 con 0x7efc7015d930 2023-06-28T20:07:14.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc7561d700 1 --2- 172.21.15.43:0/3675423085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc5c004850 0x7efc5c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:14.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc75e1e700 1 --2- 172.21.15.43:0/3675423085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc5c003bc0 0x7efc5c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:14.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc7661f700 1 --2- 172.21.15.43:0/3675423085 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc7015d930 0x7efc5c005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:14.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc7561d700 1 -- 172.21.15.43:0/3675423085 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc7015d930 msgr2=0x7efc5c005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:14.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc7561d700 1 --2- 172.21.15.43:0/3675423085 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc7015d930 0x7efc5c005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:14.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc7561d700 1 -- 172.21.15.43:0/3675423085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc5c003bc0 msgr2=0x7efc5c003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:14.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc7561d700 1 --2- 172.21.15.43:0/3675423085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc5c003bc0 0x7efc5c003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:14.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.446+0000 7efc7561d700 1 -- 172.21.15.43:0/3675423085 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc600094a0 con 0x7efc5c004850 2023-06-28T20:07:14.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.447+0000 7efc7561d700 1 --2- 172.21.15.43:0/3675423085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc5c004850 0x7efc5c004cc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00c210 tx=0x7efc5c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb92a4bac2b21ca6 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:14.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.447+0000 7efc74e1c700 1 -- 172.21.15.43:0/3675423085 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c00cd10 con 0x7efc5c004850 2023-06-28T20:07:14.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.447+0000 7efc74e1c700 1 -- 172.21.15.43:0/3675423085 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc6c00ce70 con 0x7efc5c004850 2023-06-28T20:07:14.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.447+0000 7efc74e1c700 1 -- 172.21.15.43:0/3675423085 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c005b60 con 0x7efc5c004850 2023-06-28T20:07:14.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.447+0000 7efc78082700 1 -- 172.21.15.43:0/3675423085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc5c004850 msgr2=0x7efc5c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:14.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.447+0000 7efc78082700 1 --2- 172.21.15.43:0/3675423085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc5c004850 0x7efc5c004cc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00c210 tx=0x7efc5c005880 comp rx=0 tx=0).stop 2023-06-28T20:07:14.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.447+0000 7efc7661f700 1 -- 172.21.15.43:0/3675423085 reap_dead start 2023-06-28T20:07:14.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.447+0000 7efc78082700 1 -- 172.21.15.43:0/3675423085 shutdown_connections 2023-06-28T20:07:14.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.447+0000 7efc78082700 1 -- 172.21.15.43:0/3675423085 >> 172.21.15.43:0/3675423085 conn(0x7efc700cd240 msgr2=0x7efc700cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:14.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.447+0000 7efc78082700 1 -- 172.21.15.43:0/3675423085 shutdown_connections 2023-06-28T20:07:14.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.447+0000 7efc78082700 1 -- 172.21.15.43:0/3675423085 wait complete. 2023-06-28T20:07:14.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.449+0000 7efc78082700 1 Processor -- start 2023-06-28T20:07:14.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.449+0000 7efc78082700 1 -- start start 2023-06-28T20:07:14.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.449+0000 7efc78082700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc5c004850 0x7efc700b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:14.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.449+0000 7efc78082700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc700b5700 0x7efc70150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:14.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.449+0000 7efc78082700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc70150c70 0x7efc70151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:14.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.449+0000 7efc78082700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc700cfa20 con 0x7efc700b5700 2023-06-28T20:07:14.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.449+0000 7efc78082700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc700cfcc0 con 0x7efc70150c70 2023-06-28T20:07:14.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.449+0000 7efc78082700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc700cffa0 con 0x7efc5c004850 2023-06-28T20:07:14.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.449+0000 7efc7561d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc700b5700 0x7efc70150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:14.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.449+0000 7efc7561d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc700b5700 0x7efc70150730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52568/0 (socket says 172.21.15.43:52568) 2023-06-28T20:07:14.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.450+0000 7efc7561d700 1 -- 172.21.15.43:0/118650067 learned_addr learned my addr 172.21.15.43:0/118650067 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:14.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.450+0000 7efc7661f700 1 --2- 172.21.15.43:0/118650067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc70150c70 0x7efc70151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:14.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.450+0000 7efc75e1e700 1 --2- 172.21.15.43:0/118650067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc5c004850 0x7efc700b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:14.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.450+0000 7efc7561d700 1 -- 172.21.15.43:0/118650067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc5c004850 msgr2=0x7efc700b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:14.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.450+0000 7efc7561d700 1 --2- 172.21.15.43:0/118650067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc5c004850 0x7efc700b1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:14.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.450+0000 7efc7561d700 1 -- 172.21.15.43:0/118650067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc70150c70 msgr2=0x7efc70151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:14.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.450+0000 7efc7561d700 1 --2- 172.21.15.43:0/118650067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc70150c70 0x7efc70151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:14.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.450+0000 7efc7561d700 1 -- 172.21.15.43:0/118650067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc700d0280 con 0x7efc700b5700 2023-06-28T20:07:14.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.450+0000 7efc7561d700 1 --2- 172.21.15.43:0/118650067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc700b5700 0x7efc70150730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efc6c0018b0 tx=0x7efc6c001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.450+0000 7efc667fc700 1 -- 172.21.15.43:0/118650067 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c0072d0 con 0x7efc700b5700 2023-06-28T20:07:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.450+0000 7efc667fc700 1 -- 172.21.15.43:0/118650067 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc6c007430 con 0x7efc700b5700 2023-06-28T20:07:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.450+0000 7efc667fc700 1 -- 172.21.15.43:0/118650067 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c0077e0 con 0x7efc700b5700 2023-06-28T20:07:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.450+0000 7efc78082700 1 -- 172.21.15.43:0/118650067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc700d0560 con 0x7efc700b5700 2023-06-28T20:07:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.450+0000 7efc78082700 1 -- 172.21.15.43:0/118650067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc700b2d40 con 0x7efc700b5700 2023-06-28T20:07:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.451+0000 7efc78082700 1 -- 172.21.15.43:0/118650067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc700728b0 con 0x7efc700b5700 2023-06-28T20:07:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.452+0000 7efc667fc700 1 -- 172.21.15.43:0/118650067 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7efc6c01e430 con 0x7efc700b5700 2023-06-28T20:07:14.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.452+0000 7efc667fc700 1 --2- 172.21.15.43:0/118650067 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efc5406ac50 0x7efc5406d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.452+0000 7efc667fc700 1 -- 172.21.15.43:0/118650067 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5188+0+0 (secure 0 0 0) 0x7efc6c08a100 con 0x7efc700b5700 2023-06-28T20:07:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.452+0000 7efc75e1e700 1 --2- 172.21.15.43:0/118650067 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efc5406ac50 0x7efc5406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.452+0000 7efc75e1e700 1 --2- 172.21.15.43:0/118650067 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efc5406ac50 0x7efc5406d100 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efc701584c0 tx=0x7efc60009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:14.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.457+0000 7efc667fc700 1 -- 172.21.15.43:0/118650067 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efc6c0557a0 con 0x7efc700b5700 2023-06-28T20:07:14.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:14.656+0000 7efc78082700 1 -- 172.21.15.43:0/118650067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7efc7006bd20 con 0x7efc700b5700 2023-06-28T20:07:15.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.298+0000 7efc667fc700 1 -- 172.21.15.43:0/118650067 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v16) v1 ==== 173+0+0 (secure 0 0 0) 0x7efc6c058dc0 con 0x7efc700b5700 2023-06-28T20:07:15.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.303+0000 7efc78082700 1 -- 172.21.15.43:0/118650067 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efc5406ac50 msgr2=0x7efc5406d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:15.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.303+0000 7efc78082700 1 --2- 172.21.15.43:0/118650067 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efc5406ac50 0x7efc5406d100 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efc701584c0 tx=0x7efc60009210 comp rx=0 tx=0).stop 2023-06-28T20:07:15.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.303+0000 7efc78082700 1 -- 172.21.15.43:0/118650067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc700b5700 msgr2=0x7efc70150730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:15.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.303+0000 7efc78082700 1 --2- 172.21.15.43:0/118650067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc700b5700 0x7efc70150730 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efc6c0018b0 tx=0x7efc6c001980 comp rx=0 tx=0).stop 2023-06-28T20:07:15.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.303+0000 7efc78082700 1 -- 172.21.15.43:0/118650067 shutdown_connections 2023-06-28T20:07:15.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.303+0000 7efc78082700 1 --2- 172.21.15.43:0/118650067 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc70150c70 0x7efc70151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:15.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.303+0000 7efc78082700 1 --2- 172.21.15.43:0/118650067 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efc5406ac50 0x7efc5406d100 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:15.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.303+0000 7efc78082700 1 --2- 172.21.15.43:0/118650067 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc5c004850 0x7efc700b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:15.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.303+0000 7efc78082700 1 --2- 172.21.15.43:0/118650067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc700b5700 0x7efc70150730 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:15.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.303+0000 7efc78082700 1 -- 172.21.15.43:0/118650067 >> 172.21.15.43:0/118650067 conn(0x7efc700cd240 msgr2=0x7efc70070800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:15.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.303+0000 7efc78082700 1 -- 172.21.15.43:0/118650067 shutdown_connections 2023-06-28T20:07:15.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.303+0000 7efc78082700 1 -- 172.21.15.43:0/118650067 wait complete. 2023-06-28T20:07:15.311 INFO:teuthology.orchestra.run.smithi043.stderr:enabled application 'rbd' on pool 'rbd' 2023-06-28T20:07:15.323 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-06-28T20:07:15.324 INFO:tasks.ceph.mds.a:Restarting daemon 2023-06-28T20:07:15.325 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-06-28T20:07:15.328 INFO:tasks.ceph.mds.a:Started 2023-06-28T20:07:15.329 INFO:tasks.ceph.mds.c:Restarting daemon 2023-06-28T20:07:15.329 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-06-28T20:07:15.332 INFO:tasks.ceph.mds.c:Started 2023-06-28T20:07:15.332 INFO:tasks.ceph.mds.b:Restarting daemon 2023-06-28T20:07:15.333 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-06-28T20:07:15.336 INFO:tasks.ceph.mds.b:Started 2023-06-28T20:07:15.336 INFO:tasks.ceph.mds.d:Restarting daemon 2023-06-28T20:07:15.336 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-06-28T20:07:15.340 INFO:tasks.ceph.mds.d:Started 2023-06-28T20:07:15.340 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-06-28T20:07:15.341 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:07:15.427 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2023-06-28T20:07:15.427 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2023-06-28T20:07:15.428 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2023-06-28T20:07:15.440 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2023-06-28T20:07:15.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.486+0000 7f6b50353700 1 Processor -- start 2023-06-28T20:07:15.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.486+0000 7f6b50353700 1 -- start start 2023-06-28T20:07:15.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.487+0000 7f6b50353700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6b480d0630 0x7f6b480d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:15.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.487+0000 7f6b50353700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b480cbd40 0x7f6b4806b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:15.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.487+0000 7f6b50353700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b480ccf10 con 0x7f6b4806b540 2023-06-28T20:07:15.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.487+0000 7f6b50353700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b4806b9c0 con 0x7f6b480d0630 2023-06-28T20:07:15.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.487+0000 7f6b50353700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4815cd80 con 0x7f6b480cbd40 2023-06-28T20:07:15.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.488+0000 7f6b4e0ef700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6b480d0630 0x7f6b480d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:15.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.488+0000 7f6b4e0ef700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6b480d0630 0x7f6b480d1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52618/0 (socket says 172.21.15.43:52618) 2023-06-28T20:07:15.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.488+0000 7f6b4e8f0700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6b4806b540 0x7f6b480cd680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41808/0 (socket says 172.21.15.43:41808) 2023-06-28T20:07:15.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.488+0000 7f6b4e0ef700 1 -- 172.21.15.43:0/1143758333 learned_addr learned my addr 172.21.15.43:0/1143758333 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:15.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.488+0000 7f6b4d8ee700 1 --2- 172.21.15.43:0/1143758333 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b480cbd40 0x7f6b4806b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:15.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.488+0000 7f6b4d0ed700 1 -- 172.21.15.43:0/1143758333 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3176694887 0 0) 0x7f6b480ccf10 con 0x7f6b4806b540 2023-06-28T20:07:15.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.488+0000 7f6b4d0ed700 1 -- 172.21.15.43:0/1143758333 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b340035b0 con 0x7f6b4806b540 2023-06-28T20:07:15.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.488+0000 7f6b4e0ef700 1 -- 172.21.15.43:0/1143758333 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b480cbd40 msgr2=0x7f6b4806b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:15.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.488+0000 7f6b4e0ef700 1 --2- 172.21.15.43:0/1143758333 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b480cbd40 0x7f6b4806b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:15.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.488+0000 7f6b4e0ef700 1 -- 172.21.15.43:0/1143758333 >> v1:172.21.15.43:6789/0 conn(0x7f6b4806b540 legacy=0x7f6b480cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:15.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.488+0000 7f6b4e0ef700 1 -- 172.21.15.43:0/1143758333 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b480d0a00 con 0x7f6b480d0630 2023-06-28T20:07:15.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.488+0000 7f6b4e0ef700 1 --2- 172.21.15.43:0/1143758333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b480d0630 0x7f6b480d1070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6b44006fd0 tx=0x7f6b44009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b342f93d96e94958 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:15.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.488+0000 7f6b4d0ed700 1 -- 172.21.15.43:0/1143758333 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b44005e30 con 0x7f6b480d0630 2023-06-28T20:07:15.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.488+0000 7f6b4d0ed700 1 -- 172.21.15.43:0/1143758333 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b44004030 con 0x7f6b480d0630 2023-06-28T20:07:15.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.488+0000 7f6b4d0ed700 1 -- 172.21.15.43:0/1143758333 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b440043e0 con 0x7f6b480d0630 2023-06-28T20:07:15.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.489+0000 7f6b50353700 1 -- 172.21.15.43:0/1143758333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b480d0630 msgr2=0x7f6b480d1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:15.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.489+0000 7f6b50353700 1 --2- 172.21.15.43:0/1143758333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b480d0630 0x7f6b480d1070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6b44006fd0 tx=0x7f6b44009bd0 comp rx=0 tx=0).stop 2023-06-28T20:07:15.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.489+0000 7f6b50353700 1 -- 172.21.15.43:0/1143758333 shutdown_connections 2023-06-28T20:07:15.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.489+0000 7f6b50353700 1 --2- 172.21.15.43:0/1143758333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b480d0630 0x7f6b480d1070 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:15.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.489+0000 7f6b50353700 1 --2- 172.21.15.43:0/1143758333 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b480cbd40 0x7f6b4806b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:15.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.489+0000 7f6b50353700 1 -- 172.21.15.43:0/1143758333 >> 172.21.15.43:0/1143758333 conn(0x7f6b480664b0 msgr2=0x7f6b4809a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:15.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.489+0000 7f6b50353700 1 -- 172.21.15.43:0/1143758333 shutdown_connections 2023-06-28T20:07:15.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.489+0000 7f6b50353700 1 -- 172.21.15.43:0/1143758333 wait complete. 2023-06-28T20:07:15.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.491+0000 7f6b50353700 1 Processor -- start 2023-06-28T20:07:15.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.491+0000 7f6b50353700 1 -- start start 2023-06-28T20:07:15.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.492+0000 7f6b50353700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b4806b540 0x7f6b480b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:15.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.492+0000 7f6b50353700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b480cbd40 0x7f6b48146ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:15.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.492+0000 7f6b50353700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b480d0630 0x7f6b480c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:15.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.492+0000 7f6b50353700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b480b4b60 con 0x7f6b4806b540 2023-06-28T20:07:15.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.492+0000 7f6b50353700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b480b3190 con 0x7f6b480d0630 2023-06-28T20:07:15.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.492+0000 7f6b50353700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b480b4db0 con 0x7f6b480cbd40 2023-06-28T20:07:15.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.492+0000 7f6b4e0ef700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b4806b540 0x7f6b480b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:15.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.492+0000 7f6b4e0ef700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b4806b540 0x7f6b480b2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52624/0 (socket says 172.21.15.43:52624) 2023-06-28T20:07:15.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.492+0000 7f6b4e0ef700 1 -- 172.21.15.43:0/2995103781 learned_addr learned my addr 172.21.15.43:0/2995103781 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:15.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.492+0000 7f6b4e8f0700 1 --2- 172.21.15.43:0/2995103781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b480d0630 0x7f6b480c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:15.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.492+0000 7f6b4d8ee700 1 --2- 172.21.15.43:0/2995103781 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b480cbd40 0x7f6b48146ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:15.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.492+0000 7f6b4e0ef700 1 -- 172.21.15.43:0/2995103781 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b480cbd40 msgr2=0x7f6b48146ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:15.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.492+0000 7f6b4e0ef700 1 --2- 172.21.15.43:0/2995103781 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b480cbd40 0x7f6b48146ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:15.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.492+0000 7f6b4e0ef700 1 -- 172.21.15.43:0/2995103781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b480d0630 msgr2=0x7f6b480c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:15.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.492+0000 7f6b4e0ef700 1 --2- 172.21.15.43:0/2995103781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b480d0630 0x7f6b480c7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:15.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.492+0000 7f6b4e0ef700 1 -- 172.21.15.43:0/2995103781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b44009500 con 0x7f6b4806b540 2023-06-28T20:07:15.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.493+0000 7f6b4e0ef700 1 --2- 172.21.15.43:0/2995103781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b4806b540 0x7f6b480b2c50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6b44009c10 tx=0x7f6b44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:15.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.493+0000 7f6b3effd700 1 -- 172.21.15.43:0/2995103781 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b44003d20 con 0x7f6b4806b540 2023-06-28T20:07:15.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.493+0000 7f6b3effd700 1 -- 172.21.15.43:0/2995103781 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4401c420 con 0x7f6b4806b540 2023-06-28T20:07:15.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.493+0000 7f6b3effd700 1 -- 172.21.15.43:0/2995103781 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b44003ea0 con 0x7f6b4806b540 2023-06-28T20:07:15.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.493+0000 7f6b50353700 1 -- 172.21.15.43:0/2995103781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b480b5090 con 0x7f6b4806b540 2023-06-28T20:07:15.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.493+0000 7f6b50353700 1 -- 172.21.15.43:0/2995103781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b480b5650 con 0x7f6b4806b540 2023-06-28T20:07:15.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.494+0000 7f6b50353700 1 -- 172.21.15.43:0/2995103781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b480728b0 con 0x7f6b4806b540 2023-06-28T20:07:15.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.494+0000 7f6b3effd700 1 -- 172.21.15.43:0/2995103781 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f6b4401c7d0 con 0x7f6b4806b540 2023-06-28T20:07:15.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.495+0000 7f6b3effd700 1 --2- 172.21.15.43:0/2995103781 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f6b2c06ac50 0x7f6b2c06d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:15.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.495+0000 7f6b3effd700 1 -- 172.21.15.43:0/2995103781 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6b4408a3b0 con 0x7f6b4806b540 2023-06-28T20:07:15.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.495+0000 7f6b4d8ee700 1 --2- 172.21.15.43:0/2995103781 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f6b2c06ac50 0x7f6b2c06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:15.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.495+0000 7f6b4d8ee700 1 --2- 172.21.15.43:0/2995103781 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f6b2c06ac50 0x7f6b2c06d100 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6b48157f60 tx=0x7f6b38008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:15.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.506+0000 7f6b3effd700 1 -- 172.21.15.43:0/2995103781 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6b44055a00 con 0x7f6b4806b540 2023-06-28T20:07:15.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.713+0000 7f6b50353700 1 -- 172.21.15.43:0/2995103781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b480af3b0 con 0x7f6b4806b540 2023-06-28T20:07:15.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.713+0000 7f6b3effd700 1 -- 172.21.15.43:0/2995103781 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+14605 (secure 0 0 0) 0x7f6b44082e60 con 0x7f6b4806b540 2023-06-28T20:07:15.717 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:15.717 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":16,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:07:15.292277+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:07:15.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.715+0000 7f6b50353700 1 -- 172.21.15.43:0/2995103781 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f6b2c06ac50 msgr2=0x7f6b2c06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:15.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.715+0000 7f6b50353700 1 --2- 172.21.15.43:0/2995103781 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f6b2c06ac50 0x7f6b2c06d100 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6b48157f60 tx=0x7f6b38008040 comp rx=0 tx=0).stop 2023-06-28T20:07:15.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.716+0000 7f6b50353700 1 -- 172.21.15.43:0/2995103781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b4806b540 msgr2=0x7f6b480b2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.716+0000 7f6b50353700 1 --2- 172.21.15.43:0/2995103781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b4806b540 0x7f6b480b2c50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6b44009c10 tx=0x7f6b44004750 comp rx=0 tx=0).stop 2023-06-28T20:07:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.716+0000 7f6b50353700 1 -- 172.21.15.43:0/2995103781 shutdown_connections 2023-06-28T20:07:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.716+0000 7f6b50353700 1 --2- 172.21.15.43:0/2995103781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b480d0630 0x7f6b480c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.716+0000 7f6b50353700 1 --2- 172.21.15.43:0/2995103781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b4806b540 0x7f6b480b2c50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.716+0000 7f6b50353700 1 --2- 172.21.15.43:0/2995103781 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f6b2c06ac50 0x7f6b2c06d100 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.716+0000 7f6b50353700 1 --2- 172.21.15.43:0/2995103781 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6b480cbd40 0x7f6b48146ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:15.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.716+0000 7f6b50353700 1 -- 172.21.15.43:0/2995103781 >> 172.21.15.43:0/2995103781 conn(0x7f6b480664b0 msgr2=0x7f6b4806ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:15.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.716+0000 7f6b50353700 1 -- 172.21.15.43:0/2995103781 shutdown_connections 2023-06-28T20:07:15.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.716+0000 7f6b50353700 1 -- 172.21.15.43:0/2995103781 wait complete. 2023-06-28T20:07:15.731 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-28T20:07:13.580969+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-28T20:07:14.038358+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-28T20:07:15.731 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-28T20:07:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.861+0000 7fbe1f719700 1 Processor -- start 2023-06-28T20:07:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.861+0000 7fbe1f719700 1 -- start start 2023-06-28T20:07:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.862+0000 7fbe1f719700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbe1815a5f0 0x7fbe180ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.862+0000 7fbe1f719700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe180ae950 con 0x7fbe1806e320 2023-06-28T20:07:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.862+0000 7fbe1f719700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe1815a9c0 con 0x7fbe180ad7d0 2023-06-28T20:07:15.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.862+0000 7fbe1f719700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe180add40 con 0x7fbe1815a5f0 2023-06-28T20:07:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.862+0000 7fbe1dcb6700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbe1806e320 0x7fbe180af4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41840/0 (socket says 172.21.15.43:41840) 2023-06-28T20:07:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.862+0000 7fbe1dcb6700 1 -- 172.21.15.43:0/2565958166 learned_addr learned my addr 172.21.15.43:0/2565958166 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.862+0000 7fbe1d4b5700 1 --2- 172.21.15.43:0/2565958166 >> v2:172.21.15.178:3300/0 conn(0x7fbe1815a5f0 0x7fbe180ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.863+0000 7fbe0ffff700 1 -- 172.21.15.43:0/2565958166 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3855926449 0 0) 0x7fbe180ae950 con 0x7fbe1806e320 2023-06-28T20:07:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.863+0000 7fbe0ffff700 1 -- 172.21.15.43:0/2565958166 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe040035b0 con 0x7fbe1806e320 2023-06-28T20:07:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.863+0000 7fbe0ffff700 1 -- 172.21.15.43:0/2565958166 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3940681701 0 0) 0x7fbe1815a9c0 con 0x7fbe180ad7d0 2023-06-28T20:07:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.863+0000 7fbe0ffff700 1 -- 172.21.15.43:0/2565958166 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe180ae950 con 0x7fbe180ad7d0 2023-06-28T20:07:15.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.863+0000 7fbe1d4b5700 1 -- 172.21.15.43:0/2565958166 >> v1:172.21.15.178:6789/0 conn(0x7fbe180ad7d0 legacy=0x7fbe180aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.863+0000 7fbe1d4b5700 1 -- 172.21.15.43:0/2565958166 >> v1:172.21.15.43:6789/0 conn(0x7fbe1806e320 legacy=0x7fbe180af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.863+0000 7fbe1d4b5700 1 -- 172.21.15.43:0/2565958166 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe180ae020 con 0x7fbe1815a5f0 2023-06-28T20:07:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.863+0000 7fbe1d4b5700 1 --2- 172.21.15.43:0/2565958166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe1815a5f0 0x7fbe180ce7d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009820 tx=0x7fbe08009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50cfb783a6f4b000 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.863+0000 7fbe0ffff700 1 -- 172.21.15.43:0/2565958166 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe08005e30 con 0x7fbe1815a5f0 2023-06-28T20:07:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.863+0000 7fbe0ffff700 1 -- 172.21.15.43:0/2565958166 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe08006030 con 0x7fbe1815a5f0 2023-06-28T20:07:15.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.863+0000 7fbe0ffff700 1 -- 172.21.15.43:0/2565958166 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe080063c0 con 0x7fbe1815a5f0 2023-06-28T20:07:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.864+0000 7fbe1f719700 1 -- 172.21.15.43:0/2565958166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe1815a5f0 msgr2=0x7fbe180ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.864+0000 7fbe1f719700 1 --2- 172.21.15.43:0/2565958166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe1815a5f0 0x7fbe180ce7d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009820 tx=0x7fbe08009b30 comp rx=0 tx=0).stop 2023-06-28T20:07:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.864+0000 7fbe1f719700 1 -- 172.21.15.43:0/2565958166 shutdown_connections 2023-06-28T20:07:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.864+0000 7fbe1f719700 1 --2- 172.21.15.43:0/2565958166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe1815a5f0 0x7fbe180ce7d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.864+0000 7fbe1f719700 1 -- 172.21.15.43:0/2565958166 >> 172.21.15.43:0/2565958166 conn(0x7fbe180664b0 msgr2=0x7fbe180d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:15.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.864+0000 7fbe1f719700 1 -- 172.21.15.43:0/2565958166 shutdown_connections 2023-06-28T20:07:15.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.864+0000 7fbe1f719700 1 -- 172.21.15.43:0/2565958166 wait complete. 2023-06-28T20:07:15.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.865+0000 7fbe1f719700 1 Processor -- start 2023-06-28T20:07:15.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.865+0000 7fbe1f719700 1 -- start start 2023-06-28T20:07:15.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.866+0000 7fbe1f719700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe1806e320 0x7fbe18150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:15.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.866+0000 7fbe1f719700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe180ad7d0 0x7fbe1806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.866+0000 7fbe1f719700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe1815a5f0 0x7fbe180c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.866+0000 7fbe1f719700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe180cc0e0 con 0x7fbe1815a5f0 2023-06-28T20:07:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.866+0000 7fbe1f719700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe180ccc20 con 0x7fbe1806e320 2023-06-28T20:07:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.866+0000 7fbe1f719700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe180cc2f0 con 0x7fbe180ad7d0 2023-06-28T20:07:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.866+0000 7fbe1dcb6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe1815a5f0 0x7fbe180c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.866+0000 7fbe1dcb6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe1815a5f0 0x7fbe180c4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52660/0 (socket says 172.21.15.43:52660) 2023-06-28T20:07:15.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.866+0000 7fbe1dcb6700 1 -- 172.21.15.43:0/1009680883 learned_addr learned my addr 172.21.15.43:0/1009680883 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.866+0000 7fbe1d4b5700 1 --2- 172.21.15.43:0/1009680883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe1806e320 0x7fbe18150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.866+0000 7fbe1ccb4700 1 --2- 172.21.15.43:0/1009680883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe180ad7d0 0x7fbe1806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.866+0000 7fbe1dcb6700 1 -- 172.21.15.43:0/1009680883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe180ad7d0 msgr2=0x7fbe1806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.866+0000 7fbe1dcb6700 1 --2- 172.21.15.43:0/1009680883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe180ad7d0 0x7fbe1806c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.866+0000 7fbe1dcb6700 1 -- 172.21.15.43:0/1009680883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe1806e320 msgr2=0x7fbe18150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.866+0000 7fbe1dcb6700 1 --2- 172.21.15.43:0/1009680883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe1806e320 0x7fbe18150090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:15.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.866+0000 7fbe1dcb6700 1 -- 172.21.15.43:0/1009680883 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe08009430 con 0x7fbe1815a5f0 2023-06-28T20:07:15.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.866+0000 7fbe1dcb6700 1 --2- 172.21.15.43:0/1009680883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe1815a5f0 0x7fbe180c4d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000cba0 tx=0x7fbe1000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:15.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.866+0000 7fbe0dffb700 1 -- 172.21.15.43:0/1009680883 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1000aa40 con 0x7fbe1815a5f0 2023-06-28T20:07:15.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.866+0000 7fbe0dffb700 1 -- 172.21.15.43:0/1009680883 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1000aba0 con 0x7fbe1815a5f0 2023-06-28T20:07:15.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.867+0000 7fbe1f719700 1 -- 172.21.15.43:0/1009680883 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe180cc5d0 con 0x7fbe1815a5f0 2023-06-28T20:07:15.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.867+0000 7fbe0dffb700 1 -- 172.21.15.43:0/1009680883 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe10012400 con 0x7fbe1815a5f0 2023-06-28T20:07:15.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.867+0000 7fbe1f719700 1 -- 172.21.15.43:0/1009680883 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe1806ba10 con 0x7fbe1815a5f0 2023-06-28T20:07:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.868+0000 7fbe1f719700 1 -- 172.21.15.43:0/1009680883 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe180ccd90 con 0x7fbe1815a5f0 2023-06-28T20:07:15.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.868+0000 7fbe0dffb700 1 -- 172.21.15.43:0/1009680883 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7fbe1001b3b0 con 0x7fbe1815a5f0 2023-06-28T20:07:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.869+0000 7fbe0dffb700 1 --2- 172.21.15.43:0/1009680883 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbdfc06ac50 0x7fbdfc06d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.869+0000 7fbe0dffb700 1 -- 172.21.15.43:0/1009680883 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fbe10088d00 con 0x7fbe1815a5f0 2023-06-28T20:07:15.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.873+0000 7fbe0dffb700 1 -- 172.21.15.43:0/1009680883 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbe100541e0 con 0x7fbe1815a5f0 2023-06-28T20:07:15.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.879+0000 7fbe1d4b5700 1 --2- 172.21.15.43:0/1009680883 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbdfc06ac50 0x7fbdfc06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:15.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:15.879+0000 7fbe1d4b5700 1 --2- 172.21.15.43:0/1009680883 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbdfc06ac50 0x7fbdfc06d100 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009fd0 tx=0x7fbe08008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:16.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.077+0000 7fbe1f719700 1 -- 172.21.15.43:0/1009680883 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbe180728b0 con 0x7fbe1815a5f0 2023-06-28T20:07:16.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.077+0000 7fbe0dffb700 1 -- 172.21.15.43:0/1009680883 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbe100170a0 con 0x7fbe1815a5f0 2023-06-28T20:07:16.081 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-06-28T20:07:16.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.080+0000 7fbe1f719700 1 -- 172.21.15.43:0/1009680883 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbdfc06ac50 msgr2=0x7fbdfc06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:16.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.080+0000 7fbe1f719700 1 --2- 172.21.15.43:0/1009680883 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbdfc06ac50 0x7fbdfc06d100 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009fd0 tx=0x7fbe08008070 comp rx=0 tx=0).stop 2023-06-28T20:07:16.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.080+0000 7fbe1f719700 1 -- 172.21.15.43:0/1009680883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe1815a5f0 msgr2=0x7fbe180c4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:16.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.080+0000 7fbe1f719700 1 --2- 172.21.15.43:0/1009680883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe1815a5f0 0x7fbe180c4d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000cba0 tx=0x7fbe1000ceb0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.080+0000 7fbe1f719700 1 -- 172.21.15.43:0/1009680883 shutdown_connections 2023-06-28T20:07:16.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.080+0000 7fbe1f719700 1 --2- 172.21.15.43:0/1009680883 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe1815a5f0 0x7fbe180c4d80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.080+0000 7fbe1f719700 1 --2- 172.21.15.43:0/1009680883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe1806e320 0x7fbe18150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.080+0000 7fbe1f719700 1 --2- 172.21.15.43:0/1009680883 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbdfc06ac50 0x7fbdfc06d100 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.080+0000 7fbe1f719700 1 --2- 172.21.15.43:0/1009680883 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe180ad7d0 0x7fbe1806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.080+0000 7fbe1f719700 1 -- 172.21.15.43:0/1009680883 >> 172.21.15.43:0/1009680883 conn(0x7fbe180664b0 msgr2=0x7fbe18159770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:16.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.080+0000 7fbe1f719700 1 -- 172.21.15.43:0/1009680883 shutdown_connections 2023-06-28T20:07:16.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.080+0000 7fbe1f719700 1 -- 172.21.15.43:0/1009680883 wait complete. 2023-06-28T20:07:16.095 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-28T20:07:16.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.221+0000 7f3085bd6700 1 Processor -- start 2023-06-28T20:07:16.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.221+0000 7f3085bd6700 1 -- start start 2023-06-28T20:07:16.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.222+0000 7f3085bd6700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f30800b20e0 0x7f308015d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:16.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.222+0000 7f3085bd6700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3080070b40 con 0x7f30801592d0 2023-06-28T20:07:16.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.222+0000 7f3085bd6700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3080070e20 con 0x7f30801596a0 2023-06-28T20:07:16.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.222+0000 7f3085bd6700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3080071100 con 0x7f30800b20e0 2023-06-28T20:07:16.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.222+0000 7f307f7fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f30801592d0 0x7f30800b19d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41862/0 (socket says 172.21.15.43:41862) 2023-06-28T20:07:16.226 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.222+0000 7f307f7fe700 1 -- 172.21.15.43:0/3504315246 learned_addr learned my addr 172.21.15.43:0/3504315246 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:16.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.222+0000 7f307effd700 1 --2- 172.21.15.43:0/3504315246 >> v2:172.21.15.178:3300/0 conn(0x7f30800b20e0 0x7f308015d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:16.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.222+0000 7f307e7fc700 1 -- 172.21.15.43:0/3504315246 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2543645187 0 0) 0x7f3080070e20 con 0x7f30801596a0 2023-06-28T20:07:16.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.222+0000 7f307e7fc700 1 -- 172.21.15.43:0/3504315246 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30680035b0 con 0x7f30801596a0 2023-06-28T20:07:16.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.223+0000 7f307effd700 1 -- 172.21.15.43:0/3504315246 >> v1:172.21.15.178:6789/0 conn(0x7f30801596a0 legacy=0x7f308015d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:16.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.223+0000 7f307effd700 1 -- 172.21.15.43:0/3504315246 >> v1:172.21.15.43:6789/0 conn(0x7f30801592d0 legacy=0x7f30800b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:16.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.223+0000 7f307effd700 1 -- 172.21.15.43:0/3504315246 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30800713e0 con 0x7f30800b20e0 2023-06-28T20:07:16.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.223+0000 7f307effd700 1 --2- 172.21.15.43:0/3504315246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30800b20e0 0x7f308015d070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3074009840 tx=0x7f3074009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa87518809d8e56c server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:16.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.223+0000 7f307e7fc700 1 -- 172.21.15.43:0/3504315246 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074011830 con 0x7f30800b20e0 2023-06-28T20:07:16.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.223+0000 7f307e7fc700 1 -- 172.21.15.43:0/3504315246 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3074011990 con 0x7f30800b20e0 2023-06-28T20:07:16.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.223+0000 7f307e7fc700 1 -- 172.21.15.43:0/3504315246 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074011d20 con 0x7f30800b20e0 2023-06-28T20:07:16.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.223+0000 7f3085bd6700 1 -- 172.21.15.43:0/3504315246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30800b20e0 msgr2=0x7f308015d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:16.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.223+0000 7f3085bd6700 1 --2- 172.21.15.43:0/3504315246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30800b20e0 0x7f308015d070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3074009840 tx=0x7f3074009b50 comp rx=0 tx=0).stop 2023-06-28T20:07:16.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.223+0000 7f3085bd6700 1 -- 172.21.15.43:0/3504315246 shutdown_connections 2023-06-28T20:07:16.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.223+0000 7f3085bd6700 1 --2- 172.21.15.43:0/3504315246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30800b20e0 0x7f308015d070 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.223+0000 7f3085bd6700 1 -- 172.21.15.43:0/3504315246 >> 172.21.15.43:0/3504315246 conn(0x7f308006b000 msgr2=0x7f308006b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:16.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.224+0000 7f3085bd6700 1 -- 172.21.15.43:0/3504315246 shutdown_connections 2023-06-28T20:07:16.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.224+0000 7f3085bd6700 1 -- 172.21.15.43:0/3504315246 wait complete. 2023-06-28T20:07:16.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.225+0000 7f3085bd6700 1 Processor -- start 2023-06-28T20:07:16.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.226+0000 7f3085bd6700 1 -- start start 2023-06-28T20:07:16.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.227+0000 7f3085bd6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30800b20e0 0x7f30800b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:16.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.227+0000 7f3085bd6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30801592d0 0x7f308014d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:16.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.227+0000 7f3085bd6700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30801596a0 0x7f3080159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:16.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.227+0000 7f3085bd6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308014d630 con 0x7f30801592d0 2023-06-28T20:07:16.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.227+0000 7f3085bd6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30801effa0 con 0x7f30800b20e0 2023-06-28T20:07:16.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.227+0000 7f3085bd6700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3080159fe0 con 0x7f30801596a0 2023-06-28T20:07:16.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.227+0000 7f307effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30801592d0 0x7f308014d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:16.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.227+0000 7f307effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30801592d0 0x7f308014d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52684/0 (socket says 172.21.15.43:52684) 2023-06-28T20:07:16.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.227+0000 7f307effd700 1 -- 172.21.15.43:0/3084136054 learned_addr learned my addr 172.21.15.43:0/3084136054 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:16.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.227+0000 7f307f7fe700 1 --2- 172.21.15.43:0/3084136054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30800b20e0 0x7f30800b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:16.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.227+0000 7f307ffff700 1 --2- 172.21.15.43:0/3084136054 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30801596a0 0x7f3080159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:16.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.227+0000 7f307f7fe700 1 -- 172.21.15.43:0/3084136054 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30801596a0 msgr2=0x7f3080159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:16.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.227+0000 7f307f7fe700 1 --2- 172.21.15.43:0/3084136054 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30801596a0 0x7f3080159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.227+0000 7f307f7fe700 1 -- 172.21.15.43:0/3084136054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30801592d0 msgr2=0x7f308014d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:16.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.227+0000 7f307f7fe700 1 --2- 172.21.15.43:0/3084136054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30801592d0 0x7f308014d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.227+0000 7f307f7fe700 1 -- 172.21.15.43:0/3084136054 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30740094a0 con 0x7f30800b20e0 2023-06-28T20:07:16.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.227+0000 7f307effd700 1 --2- 172.21.15.43:0/3084136054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30801592d0 0x7f308014d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:16.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.228+0000 7f307f7fe700 1 --2- 172.21.15.43:0/3084136054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30800b20e0 0x7f30800b40f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f307000c500 tx=0x7f307000c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:16.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.228+0000 7f3067fff700 1 -- 172.21.15.43:0/3084136054 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070005d70 con 0x7f30800b20e0 2023-06-28T20:07:16.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.228+0000 7f3067fff700 1 -- 172.21.15.43:0/3084136054 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30700070d0 con 0x7f30800b20e0 2023-06-28T20:07:16.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.228+0000 7f3067fff700 1 -- 172.21.15.43:0/3084136054 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070007440 con 0x7f30800b20e0 2023-06-28T20:07:16.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.228+0000 7f3085bd6700 1 -- 172.21.15.43:0/3084136054 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f308015ab20 con 0x7f30800b20e0 2023-06-28T20:07:16.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.228+0000 7f3085bd6700 1 -- 172.21.15.43:0/3084136054 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f308015a560 con 0x7f30800b20e0 2023-06-28T20:07:16.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.229+0000 7f3085bd6700 1 -- 172.21.15.43:0/3084136054 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30800cc3e0 con 0x7f30800b20e0 2023-06-28T20:07:16.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.229+0000 7f3067fff700 1 -- 172.21.15.43:0/3084136054 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f30700075a0 con 0x7f30800b20e0 2023-06-28T20:07:16.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.230+0000 7f3067fff700 1 --2- 172.21.15.43:0/3084136054 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f306c06ac50 0x7f306c06d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:16.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.230+0000 7f307effd700 1 --2- 172.21.15.43:0/3084136054 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f306c06ac50 0x7f306c06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:16.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.230+0000 7f3067fff700 1 -- 172.21.15.43:0/3084136054 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f30700882a0 con 0x7f30800b20e0 2023-06-28T20:07:16.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.230+0000 7f307effd700 1 --2- 172.21.15.43:0/3084136054 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f306c06ac50 0x7f306c06d100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3074014040 tx=0x7f307401c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:16.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.235+0000 7f3067fff700 1 -- 172.21.15.43:0/3084136054 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3070053790 con 0x7f30800b20e0 2023-06-28T20:07:16.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.439+0000 7f3085bd6700 1 -- 172.21.15.43:0/3084136054 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f30801f0170 con 0x7f30800b20e0 2023-06-28T20:07:16.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.440+0000 7f3067fff700 1 -- 172.21.15.43:0/3084136054 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3070056db0 con 0x7f30800b20e0 2023-06-28T20:07:16.443 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-06-28T20:07:16.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.442+0000 7f3085bd6700 1 -- 172.21.15.43:0/3084136054 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f306c06ac50 msgr2=0x7f306c06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:16.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.442+0000 7f3085bd6700 1 --2- 172.21.15.43:0/3084136054 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f306c06ac50 0x7f306c06d100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3074014040 tx=0x7f307401c040 comp rx=0 tx=0).stop 2023-06-28T20:07:16.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.442+0000 7f3085bd6700 1 -- 172.21.15.43:0/3084136054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30800b20e0 msgr2=0x7f30800b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:16.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.442+0000 7f3085bd6700 1 --2- 172.21.15.43:0/3084136054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30800b20e0 0x7f30800b40f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f307000c500 tx=0x7f307000c810 comp rx=0 tx=0).stop 2023-06-28T20:07:16.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.442+0000 7f3085bd6700 1 -- 172.21.15.43:0/3084136054 shutdown_connections 2023-06-28T20:07:16.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.442+0000 7f3085bd6700 1 --2- 172.21.15.43:0/3084136054 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f306c06ac50 0x7f306c06d100 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.442+0000 7f3085bd6700 1 --2- 172.21.15.43:0/3084136054 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30801596a0 0x7f3080159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.442+0000 7f3085bd6700 1 --2- 172.21.15.43:0/3084136054 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30800b20e0 0x7f30800b40f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.442+0000 7f3085bd6700 1 --2- 172.21.15.43:0/3084136054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30801592d0 0x7f308014d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.442+0000 7f3085bd6700 1 -- 172.21.15.43:0/3084136054 >> 172.21.15.43:0/3084136054 conn(0x7f308006b000 msgr2=0x7f3080154ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:16.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.442+0000 7f3085bd6700 1 -- 172.21.15.43:0/3084136054 shutdown_connections 2023-06-28T20:07:16.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.443+0000 7f3085bd6700 1 -- 172.21.15.43:0/3084136054 wait complete. 2023-06-28T20:07:16.459 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-28T20:07:16.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.580+0000 7f0a1bd05700 1 Processor -- start 2023-06-28T20:07:16.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.580+0000 7f0a1bd05700 1 -- start start 2023-06-28T20:07:16.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.581+0000 7f0a1bd05700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0a140cc600 0x7f0a1415a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:16.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.581+0000 7f0a1bd05700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0a14159820 0x7f0a140cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:16.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.581+0000 7f0a1bd05700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a1406ee20 0x7f0a140cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:16.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.581+0000 7f0a1bd05700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a140ccf10 con 0x7f0a14159820 2023-06-28T20:07:16.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.581+0000 7f0a1bd05700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a140ce0a0 con 0x7f0a140cc600 2023-06-28T20:07:16.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.581+0000 7f0a1bd05700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a140ce380 con 0x7f0a1406ee20 2023-06-28T20:07:16.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.581+0000 7f0a192a0700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0a14159820 0x7f0a140cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:16.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.581+0000 7f0a192a0700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0a14159820 0x7f0a140cc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52698/0 (socket says 172.21.15.43:52698) 2023-06-28T20:07:16.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.581+0000 7f0a192a0700 1 -- 172.21.15.43:0/4158381080 learned_addr learned my addr 172.21.15.43:0/4158381080 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:16.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.582+0000 7f0a19aa1700 1 --2- 172.21.15.43:0/4158381080 >> v2:172.21.15.178:3300/0 conn(0x7f0a140cc600 0x7f0a1415a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:16.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.582+0000 7f0a1a2a2700 1 --2- 172.21.15.43:0/4158381080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a1406ee20 0x7f0a140cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:16.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.582+0000 7f0a192a0700 1 -- 172.21.15.43:0/4158381080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a1406ee20 msgr2=0x7f0a140cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:16.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.582+0000 7f0a192a0700 1 --2- 172.21.15.43:0/4158381080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a1406ee20 0x7f0a140cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.582+0000 7f0a192a0700 1 -- 172.21.15.43:0/4158381080 >> v2:172.21.15.178:3300/0 conn(0x7f0a140cc600 msgr2=0x7f0a1415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:16.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.582+0000 7f0a192a0700 1 --2- 172.21.15.43:0/4158381080 >> v2:172.21.15.178:3300/0 conn(0x7f0a140cc600 0x7f0a1415a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.582+0000 7f0a192a0700 1 -- 172.21.15.43:0/4158381080 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a14159c90 con 0x7f0a14159820 2023-06-28T20:07:16.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.582+0000 7f0a19aa1700 1 --2- 172.21.15.43:0/4158381080 >> v2:172.21.15.178:3300/0 conn(0x7f0a140cc600 0x7f0a1415a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:16.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.582+0000 7f0a192a0700 1 --2- 172.21.15.43:0/4158381080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a14159820 0x7f0a140cc9d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0a140720f0 tx=0x7f0a0800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd56dfa1cf8fedf2 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:16.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.582+0000 7f0a18a9f700 1 -- 172.21.15.43:0/4158381080 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0800dd30 con 0x7f0a14159820 2023-06-28T20:07:16.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.582+0000 7f0a18a9f700 1 -- 172.21.15.43:0/4158381080 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0800de90 con 0x7f0a14159820 2023-06-28T20:07:16.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.582+0000 7f0a18a9f700 1 -- 172.21.15.43:0/4158381080 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08010690 con 0x7f0a14159820 2023-06-28T20:07:16.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.583+0000 7f0a1bd05700 1 -- 172.21.15.43:0/4158381080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a14159820 msgr2=0x7f0a140cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:16.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.583+0000 7f0a1bd05700 1 --2- 172.21.15.43:0/4158381080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a14159820 0x7f0a140cc9d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0a140720f0 tx=0x7f0a0800be70 comp rx=0 tx=0).stop 2023-06-28T20:07:16.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.583+0000 7f0a1bd05700 1 -- 172.21.15.43:0/4158381080 shutdown_connections 2023-06-28T20:07:16.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.583+0000 7f0a1bd05700 1 --2- 172.21.15.43:0/4158381080 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a1406ee20 0x7f0a140cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.583+0000 7f0a1bd05700 1 --2- 172.21.15.43:0/4158381080 >> v2:172.21.15.178:3300/0 conn(0x7f0a140cc600 0x7f0a1415a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.583+0000 7f0a1bd05700 1 --2- 172.21.15.43:0/4158381080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a14159820 0x7f0a140cc9d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.583+0000 7f0a1bd05700 1 -- 172.21.15.43:0/4158381080 >> 172.21.15.43:0/4158381080 conn(0x7f0a140b1720 msgr2=0x7f0a140b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:16.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.583+0000 7f0a1bd05700 1 -- 172.21.15.43:0/4158381080 shutdown_connections 2023-06-28T20:07:16.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.583+0000 7f0a1bd05700 1 -- 172.21.15.43:0/4158381080 wait complete. 2023-06-28T20:07:16.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.584+0000 7f0a1bd05700 1 Processor -- start 2023-06-28T20:07:16.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.584+0000 7f0a1bd05700 1 -- start start 2023-06-28T20:07:16.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.585+0000 7f0a1bd05700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a1406ee20 0x7f0a141514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:16.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.585+0000 7f0a1bd05700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a140cc600 0x7f0a14155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:16.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.585+0000 7f0a1bd05700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a14159820 0x7f0a140b4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:16.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.585+0000 7f0a1bd05700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a14155ae0 con 0x7f0a14159820 2023-06-28T20:07:16.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.585+0000 7f0a1bd05700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a14151a20 con 0x7f0a1406ee20 2023-06-28T20:07:16.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.585+0000 7f0a1bd05700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a140d0ad0 con 0x7f0a140cc600 2023-06-28T20:07:16.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.585+0000 7f0a1a2a2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a14159820 0x7f0a140b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:16.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.585+0000 7f0a1a2a2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a14159820 0x7f0a140b4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52708/0 (socket says 172.21.15.43:52708) 2023-06-28T20:07:16.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.585+0000 7f0a1a2a2700 1 -- 172.21.15.43:0/1590660378 learned_addr learned my addr 172.21.15.43:0/1590660378 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:16.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.585+0000 7f0a19aa1700 1 --2- 172.21.15.43:0/1590660378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a1406ee20 0x7f0a141514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:16.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.585+0000 7f0a192a0700 1 --2- 172.21.15.43:0/1590660378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a140cc600 0x7f0a14155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:16.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.585+0000 7f0a1a2a2700 1 -- 172.21.15.43:0/1590660378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a140cc600 msgr2=0x7f0a14155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:16.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.585+0000 7f0a1a2a2700 1 --2- 172.21.15.43:0/1590660378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a140cc600 0x7f0a14155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.585+0000 7f0a1a2a2700 1 -- 172.21.15.43:0/1590660378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a1406ee20 msgr2=0x7f0a141514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:16.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.585+0000 7f0a1a2a2700 1 --2- 172.21.15.43:0/1590660378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a1406ee20 0x7f0a141514e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.585+0000 7f0a1a2a2700 1 -- 172.21.15.43:0/1590660378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0800b7f0 con 0x7f0a14159820 2023-06-28T20:07:16.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.585+0000 7f0a1a2a2700 1 --2- 172.21.15.43:0/1590660378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a14159820 0x7f0a140b4750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00ccf0 tx=0x7f0a0c00a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:16.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.585+0000 7f0a067fc700 1 -- 172.21.15.43:0/1590660378 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c003a20 con 0x7f0a14159820 2023-06-28T20:07:16.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.585+0000 7f0a067fc700 1 -- 172.21.15.43:0/1590660378 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0c003b80 con 0x7f0a14159820 2023-06-28T20:07:16.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.585+0000 7f0a1bd05700 1 -- 172.21.15.43:0/1590660378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a140d07f0 con 0x7f0a14159820 2023-06-28T20:07:16.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.585+0000 7f0a067fc700 1 -- 172.21.15.43:0/1590660378 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c004020 con 0x7f0a14159820 2023-06-28T20:07:16.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.585+0000 7f0a1bd05700 1 -- 172.21.15.43:0/1590660378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a14155700 con 0x7f0a14159820 2023-06-28T20:07:16.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.586+0000 7f0a1bd05700 1 -- 172.21.15.43:0/1590660378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a14155c20 con 0x7f0a14159820 2023-06-28T20:07:16.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.587+0000 7f0a067fc700 1 -- 172.21.15.43:0/1590660378 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f0a0c004180 con 0x7f0a14159820 2023-06-28T20:07:16.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.587+0000 7f0a067fc700 1 --2- 172.21.15.43:0/1590660378 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f09f806ac50 0x7f09f806d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:16.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.587+0000 7f0a067fc700 1 -- 172.21.15.43:0/1590660378 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0a0c012070 con 0x7f0a14159820 2023-06-28T20:07:16.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.587+0000 7f0a19aa1700 1 --2- 172.21.15.43:0/1590660378 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f09f806ac50 0x7f09f806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:16.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.588+0000 7f0a19aa1700 1 --2- 172.21.15.43:0/1590660378 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f09f806ac50 0x7f09f806d100 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0a10006fd0 tx=0x7f0a10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:16.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.592+0000 7f0a067fc700 1 -- 172.21.15.43:0/1590660378 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0a14155c20 con 0x7f0a14159820 2023-06-28T20:07:16.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.832+0000 7f0a1bd05700 1 -- 172.21.15.43:0/1590660378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a14155c20 con 0x7f0a14159820 2023-06-28T20:07:16.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.833+0000 7f0a067fc700 1 -- 172.21.15.43:0/1590660378 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3524 (secure 0 0 0) 0x7f0a14155c20 con 0x7f0a14159820 2023-06-28T20:07:16.836 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:16.836 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4033115951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4033115951},{"type":"v1","addr":"172.21.15.178:6835","nonce":4033115951}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/65569869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":65569869},{"type":"v1","addr":"172.21.15.43:6837","nonce":65569869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/1016213057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016213057},{"type":"v1","addr":"172.21.15.178:6834","nonce":1016213057}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4300,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3514544484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3514544484},{"type":"v1","addr":"172.21.15.43:6835","nonce":3514544484}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-06-28T20:07:16.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.835+0000 7f0a1bd05700 1 -- 172.21.15.43:0/1590660378 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f09f806ac50 msgr2=0x7f09f806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:16.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.835+0000 7f0a1bd05700 1 --2- 172.21.15.43:0/1590660378 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f09f806ac50 0x7f09f806d100 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0a10006fd0 tx=0x7f0a10008040 comp rx=0 tx=0).stop 2023-06-28T20:07:16.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.835+0000 7f0a1bd05700 1 -- 172.21.15.43:0/1590660378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a14159820 msgr2=0x7f0a140b4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:16.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.835+0000 7f0a1bd05700 1 --2- 172.21.15.43:0/1590660378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a14159820 0x7f0a140b4750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00ccf0 tx=0x7f0a0c00a570 comp rx=0 tx=0).stop 2023-06-28T20:07:16.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.835+0000 7f0a1bd05700 1 -- 172.21.15.43:0/1590660378 shutdown_connections 2023-06-28T20:07:16.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.835+0000 7f0a1bd05700 1 --2- 172.21.15.43:0/1590660378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a14159820 0x7f0a140b4750 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.835+0000 7f0a1bd05700 1 --2- 172.21.15.43:0/1590660378 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a1406ee20 0x7f0a141514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.835+0000 7f0a1bd05700 1 --2- 172.21.15.43:0/1590660378 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f09f806ac50 0x7f09f806d100 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.835+0000 7f0a1bd05700 1 --2- 172.21.15.43:0/1590660378 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a140cc600 0x7f0a14155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.835+0000 7f0a1bd05700 1 -- 172.21.15.43:0/1590660378 >> 172.21.15.43:0/1590660378 conn(0x7f0a140b1720 msgr2=0x7f0a14152ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:16.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.835+0000 7f0a1bd05700 1 -- 172.21.15.43:0/1590660378 shutdown_connections 2023-06-28T20:07:16.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.835+0000 7f0a1bd05700 1 -- 172.21.15.43:0/1590660378 wait complete. 2023-06-28T20:07:16.840 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 3 2023-06-28T20:07:16.850 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:07:16.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.976+0000 7f5a076d6700 1 Processor -- start 2023-06-28T20:07:16.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.976+0000 7f5a076d6700 1 -- start start 2023-06-28T20:07:16.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.976+0000 7f5a076d6700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5a000cc600 0x7f5a0015a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:16.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.976+0000 7f5a076d6700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a00159820 0x7f5a000cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:16.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.976+0000 7f5a076d6700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5a0006ee20 0x7f5a000cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:16.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.977+0000 7f5a076d6700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a000ccf10 con 0x7f5a0006ee20 2023-06-28T20:07:16.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.977+0000 7f5a076d6700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a000ce0a0 con 0x7f5a000cc600 2023-06-28T20:07:16.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.977+0000 7f5a076d6700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a000ce380 con 0x7f5a00159820 2023-06-28T20:07:16.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.977+0000 7f5a05c73700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5a0006ee20 0x7f5a000cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:16.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.977+0000 7f5a05c73700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5a0006ee20 0x7f5a000cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52714/0 (socket says 172.21.15.43:52714) 2023-06-28T20:07:16.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.977+0000 7f5a05c73700 1 -- 172.21.15.43:0/414434097 learned_addr learned my addr 172.21.15.43:0/414434097 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:16.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.977+0000 7f5a04c71700 1 --2- 172.21.15.43:0/414434097 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a00159820 0x7f5a000cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:16.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.977+0000 7f5a05472700 1 --2- 172.21.15.43:0/414434097 >> v2:172.21.15.178:3300/0 conn(0x7f5a000cc600 0x7f5a0015a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:16.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.977+0000 7f5a05c73700 1 -- 172.21.15.43:0/414434097 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a00159820 msgr2=0x7f5a000cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:16.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.977+0000 7f5a05c73700 1 --2- 172.21.15.43:0/414434097 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a00159820 0x7f5a000cc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.977+0000 7f5a05c73700 1 -- 172.21.15.43:0/414434097 >> v2:172.21.15.178:3300/0 conn(0x7f5a000cc600 msgr2=0x7f5a0015a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:16.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.977+0000 7f5a05c73700 1 --2- 172.21.15.43:0/414434097 >> v2:172.21.15.178:3300/0 conn(0x7f5a000cc600 0x7f5a0015a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.977+0000 7f5a05c73700 1 -- 172.21.15.43:0/414434097 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a00159c90 con 0x7f5a0006ee20 2023-06-28T20:07:16.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.977+0000 7f5a05c73700 1 --2- 172.21.15.43:0/414434097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a0006ee20 0x7f5a000cd7e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5a000720f0 tx=0x7f59f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a29c21cae84aca server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:16.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.977+0000 7f59f7fff700 1 -- 172.21.15.43:0/414434097 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f800e070 con 0x7f5a0006ee20 2023-06-28T20:07:16.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.977+0000 7f59f7fff700 1 -- 172.21.15.43:0/414434097 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f800b9c0 con 0x7f5a0006ee20 2023-06-28T20:07:16.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.977+0000 7f59f7fff700 1 -- 172.21.15.43:0/414434097 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f800bd70 con 0x7f5a0006ee20 2023-06-28T20:07:16.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.978+0000 7f5a076d6700 1 -- 172.21.15.43:0/414434097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a0006ee20 msgr2=0x7f5a000cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:16.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.978+0000 7f5a076d6700 1 --2- 172.21.15.43:0/414434097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a0006ee20 0x7f5a000cd7e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5a000720f0 tx=0x7f59f800b330 comp rx=0 tx=0).stop 2023-06-28T20:07:16.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.978+0000 7f5a076d6700 1 -- 172.21.15.43:0/414434097 shutdown_connections 2023-06-28T20:07:16.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.978+0000 7f5a076d6700 1 --2- 172.21.15.43:0/414434097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a0006ee20 0x7f5a000cd7e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.978+0000 7f5a076d6700 1 --2- 172.21.15.43:0/414434097 >> v2:172.21.15.178:3300/0 conn(0x7f5a000cc600 0x7f5a0015a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.978+0000 7f5a076d6700 1 --2- 172.21.15.43:0/414434097 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a00159820 0x7f5a000cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.978+0000 7f5a076d6700 1 -- 172.21.15.43:0/414434097 >> 172.21.15.43:0/414434097 conn(0x7f5a000b1720 msgr2=0x7f5a000b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:16.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.978+0000 7f5a076d6700 1 -- 172.21.15.43:0/414434097 shutdown_connections 2023-06-28T20:07:16.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.978+0000 7f5a076d6700 1 -- 172.21.15.43:0/414434097 wait complete. 2023-06-28T20:07:16.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.979+0000 7f5a076d6700 1 Processor -- start 2023-06-28T20:07:16.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.979+0000 7f5a076d6700 1 -- start start 2023-06-28T20:07:16.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.980+0000 7f5a076d6700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a0006ee20 0x7f5a00144bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:16.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.980+0000 7f5a076d6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a000cc600 0x7f5a0014da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:16.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.980+0000 7f5a076d6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a00159820 0x7f5a0014dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:16.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.980+0000 7f5a076d6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a001453d0 con 0x7f5a000cc600 2023-06-28T20:07:16.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.980+0000 7f5a076d6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a001499b0 con 0x7f5a00159820 2023-06-28T20:07:16.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.980+0000 7f5a076d6700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a00148de0 con 0x7f5a0006ee20 2023-06-28T20:07:16.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.980+0000 7f5a04c71700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a000cc600 0x7f5a0014da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:16.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.980+0000 7f5a04c71700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a000cc600 0x7f5a0014da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52720/0 (socket says 172.21.15.43:52720) 2023-06-28T20:07:16.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.980+0000 7f5a04c71700 1 -- 172.21.15.43:0/3943294166 learned_addr learned my addr 172.21.15.43:0/3943294166 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:16.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.980+0000 7f5a05472700 1 --2- 172.21.15.43:0/3943294166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a0006ee20 0x7f5a00144bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:16.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.980+0000 7f5a05c73700 1 --2- 172.21.15.43:0/3943294166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a00159820 0x7f5a0014dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:16.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.980+0000 7f5a04c71700 1 -- 172.21.15.43:0/3943294166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a0006ee20 msgr2=0x7f5a00144bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:16.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.980+0000 7f5a04c71700 1 --2- 172.21.15.43:0/3943294166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a0006ee20 0x7f5a00144bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.980+0000 7f5a04c71700 1 -- 172.21.15.43:0/3943294166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a00159820 msgr2=0x7f5a0014dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:16.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.980+0000 7f5a04c71700 1 --2- 172.21.15.43:0/3943294166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a00159820 0x7f5a0014dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:16.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.980+0000 7f5a04c71700 1 -- 172.21.15.43:0/3943294166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f8009b30 con 0x7f5a000cc600 2023-06-28T20:07:16.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.980+0000 7f5a05c73700 1 --2- 172.21.15.43:0/3943294166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a00159820 0x7f5a0014dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:16.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.980+0000 7f5a04c71700 1 --2- 172.21.15.43:0/3943294166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a000cc600 0x7f5a0014da90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00d800 tx=0x7f59fc00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:16.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.981+0000 7f59f5ffb700 1 -- 172.21.15.43:0/3943294166 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc0125f0 con 0x7f5a000cc600 2023-06-28T20:07:16.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.981+0000 7f59f5ffb700 1 -- 172.21.15.43:0/3943294166 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59fc012750 con 0x7f5a000cc600 2023-06-28T20:07:16.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.981+0000 7f59f5ffb700 1 -- 172.21.15.43:0/3943294166 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc012b00 con 0x7f5a000cc600 2023-06-28T20:07:16.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.981+0000 7f5a076d6700 1 -- 172.21.15.43:0/3943294166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a00149080 con 0x7f5a000cc600 2023-06-28T20:07:16.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.981+0000 7f5a076d6700 1 -- 172.21.15.43:0/3943294166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a00149640 con 0x7f5a000cc600 2023-06-28T20:07:16.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.982+0000 7f5a076d6700 1 -- 172.21.15.43:0/3943294166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a00149b60 con 0x7f5a000cc600 2023-06-28T20:07:16.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.982+0000 7f59f5ffb700 1 -- 172.21.15.43:0/3943294166 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f59fc012c60 con 0x7f5a000cc600 2023-06-28T20:07:16.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.982+0000 7f59f5ffb700 1 --2- 172.21.15.43:0/3943294166 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f59e406ace0 0x7f59e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:16.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.982+0000 7f59f5ffb700 1 -- 172.21.15.43:0/3943294166 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f59fc089430 con 0x7f5a000cc600 2023-06-28T20:07:16.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.982+0000 7f5a05472700 1 --2- 172.21.15.43:0/3943294166 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f59e406ace0 0x7f59e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:16.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.983+0000 7f5a05472700 1 --2- 172.21.15.43:0/3943294166 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f59e406ace0 0x7f59e406d190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009950 tx=0x7f59f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:16.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:16.987+0000 7f59f5ffb700 1 -- 172.21.15.43:0/3943294166 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f59fc054830 con 0x7f5a000cc600 2023-06-28T20:07:17.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.184+0000 7f5a076d6700 1 -- 172.21.15.43:0/3943294166 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a001497d0 con 0x7f5a000cc600 2023-06-28T20:07:17.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.185+0000 7f59f5ffb700 1 -- 172.21.15.43:0/3943294166 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+14579 (secure 0 0 0) 0x7f59fc057e50 con 0x7f5a000cc600 2023-06-28T20:07:17.188 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:17.188 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":17,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:07:16.298016+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:07:17.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.187+0000 7f5a076d6700 1 -- 172.21.15.43:0/3943294166 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f59e406ace0 msgr2=0x7f59e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:17.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.187+0000 7f5a076d6700 1 --2- 172.21.15.43:0/3943294166 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f59e406ace0 0x7f59e406d190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009950 tx=0x7f59f0008040 comp rx=0 tx=0).stop 2023-06-28T20:07:17.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.187+0000 7f5a076d6700 1 -- 172.21.15.43:0/3943294166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a000cc600 msgr2=0x7f5a0014da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:17.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.187+0000 7f5a076d6700 1 --2- 172.21.15.43:0/3943294166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a000cc600 0x7f5a0014da90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00d800 tx=0x7f59fc00db10 comp rx=0 tx=0).stop 2023-06-28T20:07:17.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.187+0000 7f5a076d6700 1 -- 172.21.15.43:0/3943294166 shutdown_connections 2023-06-28T20:07:17.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.187+0000 7f5a076d6700 1 --2- 172.21.15.43:0/3943294166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a00159820 0x7f5a0014dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:17.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.187+0000 7f5a076d6700 1 --2- 172.21.15.43:0/3943294166 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f59e406ace0 0x7f59e406d190 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:17.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.187+0000 7f5a076d6700 1 --2- 172.21.15.43:0/3943294166 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a0006ee20 0x7f5a00144bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:17.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.187+0000 7f5a076d6700 1 --2- 172.21.15.43:0/3943294166 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a000cc600 0x7f5a0014da90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:17.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.187+0000 7f5a076d6700 1 -- 172.21.15.43:0/3943294166 >> 172.21.15.43:0/3943294166 conn(0x7f5a000b1720 msgr2=0x7f5a000ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:17.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.187+0000 7f5a076d6700 1 -- 172.21.15.43:0/3943294166 shutdown_connections 2023-06-28T20:07:17.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.187+0000 7f5a076d6700 1 -- 172.21.15.43:0/3943294166 wait complete. 2023-06-28T20:07:17.202 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-28T20:07:13.580969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-28T20:07:14.038358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-28T20:07:17.203 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-28T20:07:17.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.327+0000 7f6439135700 1 Processor -- start 2023-06-28T20:07:17.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.327+0000 7f6439135700 1 -- start start 2023-06-28T20:07:17.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.328+0000 7f6439135700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64340ceee0 0x7f643406d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:17.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.328+0000 7f6439135700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64340ae950 con 0x7f643415a5f0 2023-06-28T20:07:17.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.328+0000 7f6439135700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643415a9c0 con 0x7f64340afdf0 2023-06-28T20:07:17.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.328+0000 7f6439135700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64340add40 con 0x7f64340ceee0 2023-06-28T20:07:17.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.328+0000 7f6432d9d700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f643415a5f0 0x7f64340ce7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41912/0 (socket says 172.21.15.43:41912) 2023-06-28T20:07:17.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.328+0000 7f6432d9d700 1 -- 172.21.15.43:0/3534786274 learned_addr learned my addr 172.21.15.43:0/3534786274 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:17.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f643259c700 1 --2- 172.21.15.43:0/3534786274 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64340ceee0 0x7f643406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:17.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f6431d9b700 1 -- 172.21.15.43:0/3534786274 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1229182878 0 0) 0x7f64340ae950 con 0x7f643415a5f0 2023-06-28T20:07:17.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f6431d9b700 1 -- 172.21.15.43:0/3534786274 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64200035b0 con 0x7f643415a5f0 2023-06-28T20:07:17.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f6431d9b700 1 -- 172.21.15.43:0/3534786274 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3835112009 0 0) 0x7f643415a9c0 con 0x7f64340afdf0 2023-06-28T20:07:17.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f6431d9b700 1 -- 172.21.15.43:0/3534786274 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64340ae950 con 0x7f64340afdf0 2023-06-28T20:07:17.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f6431d9b700 1 -- 172.21.15.43:0/3534786274 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 162350396 0 0) 0x7f64200035b0 con 0x7f643415a5f0 2023-06-28T20:07:17.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f6431d9b700 1 -- 172.21.15.43:0/3534786274 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64340ceee0 msgr2=0x7f643406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:17.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f6431d9b700 1 --2- 172.21.15.43:0/3534786274 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64340ceee0 0x7f643406d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:17.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f6431d9b700 1 -- 172.21.15.43:0/3534786274 >> v1:172.21.15.178:6789/0 conn(0x7f64340afdf0 legacy=0x7f643406d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:17.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f6431d9b700 1 -- 172.21.15.43:0/3534786274 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64340ae020 con 0x7f643415a5f0 2023-06-28T20:07:17.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f6431d9b700 1 -- 172.21.15.43:0/3534786274 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f641c0017f0 con 0x7f643415a5f0 2023-06-28T20:07:17.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f643259c700 1 --2- 172.21.15.43:0/3534786274 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64340ceee0 0x7f643406d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:17.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f6431d9b700 1 -- 172.21.15.43:0/3534786274 >> v1:172.21.15.43:6789/0 conn(0x7f643415a5f0 legacy=0x7f64340ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:17.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f6431d9b700 1 --2- 172.21.15.43:0/3534786274 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64340ceee0 0x7f6420003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:17.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f6431d9b700 1 --2- 172.21.15.43:0/3534786274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6420004100 0x7f6420004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:17.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f6431d9b700 1 --2- 172.21.15.43:0/3534786274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6420004d90 0x7f6420005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:17.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f6431d9b700 1 -- 172.21.15.43:0/3534786274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64340add40 con 0x7f6420004d90 2023-06-28T20:07:17.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f6431d9b700 1 -- 172.21.15.43:0/3534786274 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642000bac0 con 0x7f6420004100 2023-06-28T20:07:17.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f6431d9b700 1 -- 172.21.15.43:0/3534786274 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f642000bc80 con 0x7f64340ceee0 2023-06-28T20:07:17.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f643359e700 1 --2- 172.21.15.43:0/3534786274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6420004d90 0x7f6420005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:17.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f6432d9d700 1 --2- 172.21.15.43:0/3534786274 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64340ceee0 0x7f6420003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:17.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f643259c700 1 --2- 172.21.15.43:0/3534786274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6420004100 0x7f6420004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:17.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f643359e700 1 -- 172.21.15.43:0/3534786274 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64340ceee0 msgr2=0x7f6420003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:17.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f643359e700 1 --2- 172.21.15.43:0/3534786274 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64340ceee0 0x7f6420003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:17.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f643359e700 1 -- 172.21.15.43:0/3534786274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6420004100 msgr2=0x7f6420004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:17.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f643359e700 1 --2- 172.21.15.43:0/3534786274 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6420004100 0x7f6420004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:17.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.329+0000 7f643359e700 1 -- 172.21.15.43:0/3534786274 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64240094a0 con 0x7f6420004d90 2023-06-28T20:07:17.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.330+0000 7f643359e700 1 --2- 172.21.15.43:0/3534786274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6420004d90 0x7f6420005200 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f642c00c490 tx=0x7f6420005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396adcaac7d008a0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:17.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.330+0000 7f6431d9b700 1 -- 172.21.15.43:0/3534786274 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c003a00 con 0x7f6420004d90 2023-06-28T20:07:17.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.330+0000 7f6431d9b700 1 -- 172.21.15.43:0/3534786274 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642c003b80 con 0x7f6420004d90 2023-06-28T20:07:17.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.330+0000 7f6431d9b700 1 -- 172.21.15.43:0/3534786274 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c0050d0 con 0x7f6420004d90 2023-06-28T20:07:17.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.330+0000 7f6439135700 1 -- 172.21.15.43:0/3534786274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6420004d90 msgr2=0x7f6420005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:17.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.330+0000 7f6439135700 1 --2- 172.21.15.43:0/3534786274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6420004d90 0x7f6420005200 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f642c00c490 tx=0x7f6420005880 comp rx=0 tx=0).stop 2023-06-28T20:07:17.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.330+0000 7f643359e700 1 -- 172.21.15.43:0/3534786274 reap_dead start 2023-06-28T20:07:17.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.330+0000 7f6439135700 1 -- 172.21.15.43:0/3534786274 shutdown_connections 2023-06-28T20:07:17.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.330+0000 7f6439135700 1 -- 172.21.15.43:0/3534786274 >> 172.21.15.43:0/3534786274 conn(0x7f64340664b0 msgr2=0x7f64340d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:17.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.330+0000 7f6439135700 1 -- 172.21.15.43:0/3534786274 shutdown_connections 2023-06-28T20:07:17.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.330+0000 7f6439135700 1 -- 172.21.15.43:0/3534786274 wait complete. 2023-06-28T20:07:17.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.332+0000 7f6439135700 1 Processor -- start 2023-06-28T20:07:17.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.332+0000 7f6439135700 1 -- start start 2023-06-28T20:07:17.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.333+0000 7f6439135700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64341ef3f0 0x7f6434159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:17.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.333+0000 7f6439135700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64341ef7c0 0x7f64341595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:17.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.333+0000 7f6439135700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f643409a9c0 0x7f643409ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:17.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.333+0000 7f6439135700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64340b1b30 con 0x7f64341ef7c0 2023-06-28T20:07:17.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.333+0000 7f6439135700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64341f31c0 con 0x7f64341ef3f0 2023-06-28T20:07:17.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.333+0000 7f6439135700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64341f34a0 con 0x7f643409a9c0 2023-06-28T20:07:17.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.333+0000 7f643259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64341ef7c0 0x7f64341595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:17.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.333+0000 7f643259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64341ef7c0 0x7f64341595d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52740/0 (socket says 172.21.15.43:52740) 2023-06-28T20:07:17.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.333+0000 7f643259c700 1 -- 172.21.15.43:0/2029673465 learned_addr learned my addr 172.21.15.43:0/2029673465 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:17.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.334+0000 7f6432d9d700 1 --2- 172.21.15.43:0/2029673465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64341ef3f0 0x7f6434159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:17.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.334+0000 7f643359e700 1 --2- 172.21.15.43:0/2029673465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f643409a9c0 0x7f643409ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:17.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.334+0000 7f643259c700 1 -- 172.21.15.43:0/2029673465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f643409a9c0 msgr2=0x7f643409ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:17.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.334+0000 7f643259c700 1 --2- 172.21.15.43:0/2029673465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f643409a9c0 0x7f643409ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:17.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.334+0000 7f643259c700 1 -- 172.21.15.43:0/2029673465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64341ef3f0 msgr2=0x7f6434159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.334+0000 7f643259c700 1 --2- 172.21.15.43:0/2029673465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64341ef3f0 0x7f6434159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.334+0000 7f643259c700 1 -- 172.21.15.43:0/2029673465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64341f3780 con 0x7f64341ef7c0 2023-06-28T20:07:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.334+0000 7f643259c700 1 --2- 172.21.15.43:0/2029673465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64341ef7c0 0x7f64341595d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6424011cd0 tx=0x7f642400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.334+0000 7f642b7fe700 1 -- 172.21.15.43:0/2029673465 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424011e70 con 0x7f64341ef7c0 2023-06-28T20:07:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.334+0000 7f642b7fe700 1 -- 172.21.15.43:0/2029673465 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642400fd40 con 0x7f64341ef7c0 2023-06-28T20:07:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.334+0000 7f6439135700 1 -- 172.21.15.43:0/2029673465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64341f3a60 con 0x7f64341ef7c0 2023-06-28T20:07:17.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.334+0000 7f642b7fe700 1 -- 172.21.15.43:0/2029673465 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424014680 con 0x7f64341ef7c0 2023-06-28T20:07:17.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.334+0000 7f6439135700 1 -- 172.21.15.43:0/2029673465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643409b3a0 con 0x7f64341ef7c0 2023-06-28T20:07:17.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.335+0000 7f6439135700 1 -- 172.21.15.43:0/2029673465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64340728b0 con 0x7f64341ef7c0 2023-06-28T20:07:17.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.335+0000 7f642b7fe700 1 -- 172.21.15.43:0/2029673465 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f64240147e0 con 0x7f64341ef7c0 2023-06-28T20:07:17.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.336+0000 7f642b7fe700 1 --2- 172.21.15.43:0/2029673465 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f641406ace0 0x7f641406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:17.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.336+0000 7f642b7fe700 1 -- 172.21.15.43:0/2029673465 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6424089990 con 0x7f64341ef7c0 2023-06-28T20:07:17.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.336+0000 7f6432d9d700 1 --2- 172.21.15.43:0/2029673465 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f641406ace0 0x7f641406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:17.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.336+0000 7f6432d9d700 1 --2- 172.21.15.43:0/2029673465 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f641406ace0 0x7f641406d190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f641c000fd0 tx=0x7f641c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:17.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.340+0000 7f642b7fe700 1 -- 172.21.15.43:0/2029673465 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6424054e20 con 0x7f64341ef7c0 2023-06-28T20:07:17.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.537+0000 7f6439135700 1 -- 172.21.15.43:0/2029673465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6434070f10 con 0x7f64341ef7c0 2023-06-28T20:07:17.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.537+0000 7f642b7fe700 1 -- 172.21.15.43:0/2029673465 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f64240220a0 con 0x7f64341ef7c0 2023-06-28T20:07:17.541 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-06-28T20:07:17.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.539+0000 7f6439135700 1 -- 172.21.15.43:0/2029673465 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f641406ace0 msgr2=0x7f641406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:17.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.539+0000 7f6439135700 1 --2- 172.21.15.43:0/2029673465 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f641406ace0 0x7f641406d190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f641c000fd0 tx=0x7f641c009040 comp rx=0 tx=0).stop 2023-06-28T20:07:17.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.539+0000 7f6439135700 1 -- 172.21.15.43:0/2029673465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64341ef7c0 msgr2=0x7f64341595d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:17.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.539+0000 7f6439135700 1 --2- 172.21.15.43:0/2029673465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64341ef7c0 0x7f64341595d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6424011cd0 tx=0x7f642400f520 comp rx=0 tx=0).stop 2023-06-28T20:07:17.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.539+0000 7f6439135700 1 -- 172.21.15.43:0/2029673465 shutdown_connections 2023-06-28T20:07:17.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.539+0000 7f6439135700 1 --2- 172.21.15.43:0/2029673465 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f641406ace0 0x7f641406d190 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:17.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.539+0000 7f6439135700 1 --2- 172.21.15.43:0/2029673465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f643409a9c0 0x7f643409ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:17.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.539+0000 7f6439135700 1 --2- 172.21.15.43:0/2029673465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64341ef3f0 0x7f6434159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:17.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.539+0000 7f6439135700 1 --2- 172.21.15.43:0/2029673465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64341ef7c0 0x7f64341595d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:17.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.539+0000 7f6439135700 1 -- 172.21.15.43:0/2029673465 >> 172.21.15.43:0/2029673465 conn(0x7f64340664b0 msgr2=0x7f64340b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:17.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.539+0000 7f6439135700 1 -- 172.21.15.43:0/2029673465 shutdown_connections 2023-06-28T20:07:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.539+0000 7f6439135700 1 -- 172.21.15.43:0/2029673465 wait complete. 2023-06-28T20:07:17.555 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-28T20:07:17.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.679+0000 7ff8fcb69700 1 Processor -- start 2023-06-28T20:07:17.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.679+0000 7ff8fcb69700 1 -- start start 2023-06-28T20:07:17.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.680+0000 7ff8fcb69700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8f80aec90 0x7ff8f8160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:17.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.680+0000 7ff8fcb69700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff8f80af060 0x7ff8f8070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:17.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.680+0000 7ff8fcb69700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f8160250 con 0x7ff8f806ccc0 2023-06-28T20:07:17.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.680+0000 7ff8fcb69700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f815cb30 con 0x7ff8f80af060 2023-06-28T20:07:17.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.680+0000 7ff8fcb69700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f8071410 con 0x7ff8f80aec90 2023-06-28T20:07:17.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.681+0000 7ff8f5d9b700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff8f80af060 0x7ff8f8070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:17.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.681+0000 7ff8f659c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8f80aec90 0x7ff8f8160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:17.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.681+0000 7ff8f6d9d700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7ff8f806ccc0 0x7ff8f80b1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:41060/0 (socket says 172.21.15.43:41060) 2023-06-28T20:07:17.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.681+0000 7ff8f6d9d700 1 -- 172.21.15.43:0/3165357623 learned_addr learned my addr 172.21.15.43:0/3165357623 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:17.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.681+0000 7ff8f5d9b700 1 -- 172.21.15.43:0/3165357623 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8f80aec90 msgr2=0x7ff8f8160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:17.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.681+0000 7ff8f5d9b700 1 --2- 172.21.15.43:0/3165357623 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8f80aec90 0x7ff8f8160cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:17.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.681+0000 7ff8f5d9b700 1 -- 172.21.15.43:0/3165357623 >> v1:172.21.15.178:6789/0 conn(0x7ff8f806ccc0 legacy=0x7ff8f80b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:17.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.681+0000 7ff8f5d9b700 1 -- 172.21.15.43:0/3165357623 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f81604f0 con 0x7ff8f80af060 2023-06-28T20:07:17.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.681+0000 7ff8f559a700 1 -- 172.21.15.43:0/3165357623 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1731613271 0 0) 0x7ff8f8160250 con 0x7ff8f806ccc0 2023-06-28T20:07:17.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.681+0000 7ff8f659c700 1 --2- 172.21.15.43:0/3165357623 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8f80aec90 0x7ff8f8160cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:17.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.681+0000 7ff8f5d9b700 1 --2- 172.21.15.43:0/3165357623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8f80af060 0x7ff8f8070bc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009840 tx=0x7ff8e8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18121f883b60514d server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:17.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.681+0000 7ff8f559a700 1 -- 172.21.15.43:0/3165357623 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8011830 con 0x7ff8f80af060 2023-06-28T20:07:17.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.682+0000 7ff8f559a700 1 -- 172.21.15.43:0/3165357623 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e8011990 con 0x7ff8f80af060 2023-06-28T20:07:17.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.682+0000 7ff8f559a700 1 -- 172.21.15.43:0/3165357623 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8011d20 con 0x7ff8f80af060 2023-06-28T20:07:17.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.682+0000 7ff8fcb69700 1 -- 172.21.15.43:0/3165357623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8f80af060 msgr2=0x7ff8f8070bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:17.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.682+0000 7ff8fcb69700 1 --2- 172.21.15.43:0/3165357623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8f80af060 0x7ff8f8070bc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009840 tx=0x7ff8e8009b50 comp rx=0 tx=0).stop 2023-06-28T20:07:17.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.682+0000 7ff8fcb69700 1 -- 172.21.15.43:0/3165357623 shutdown_connections 2023-06-28T20:07:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.682+0000 7ff8fcb69700 1 --2- 172.21.15.43:0/3165357623 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8f80aec90 0x7ff8f8160cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.682+0000 7ff8fcb69700 1 --2- 172.21.15.43:0/3165357623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8f80af060 0x7ff8f8070bc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.682+0000 7ff8fcb69700 1 -- 172.21.15.43:0/3165357623 >> 172.21.15.43:0/3165357623 conn(0x7ff8f806b000 msgr2=0x7ff8f806b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.682+0000 7ff8fcb69700 1 -- 172.21.15.43:0/3165357623 shutdown_connections 2023-06-28T20:07:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.682+0000 7ff8fcb69700 1 -- 172.21.15.43:0/3165357623 wait complete. 2023-06-28T20:07:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.683+0000 7ff8fcb69700 1 Processor -- start 2023-06-28T20:07:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.684+0000 7ff8fcb69700 1 -- start start 2023-06-28T20:07:17.690 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.684+0000 7ff8fcb69700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8f806ccc0 0x7ff8f8150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:17.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.684+0000 7ff8fcb69700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8f80aec90 0x7ff8f80b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:17.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.684+0000 7ff8fcb69700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8f80af060 0x7ff8f80b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:17.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.684+0000 7ff8fcb69700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f80b3df0 con 0x7ff8f806ccc0 2023-06-28T20:07:17.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.684+0000 7ff8fcb69700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f80c3350 con 0x7ff8f80aec90 2023-06-28T20:07:17.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.684+0000 7ff8fcb69700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f80b2e80 con 0x7ff8f80af060 2023-06-28T20:07:17.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.684+0000 7ff8f659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8f806ccc0 0x7ff8f8150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:17.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.684+0000 7ff8f659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8f806ccc0 0x7ff8f8150c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52752/0 (socket says 172.21.15.43:52752) 2023-06-28T20:07:17.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.684+0000 7ff8f659c700 1 -- 172.21.15.43:0/2170987878 learned_addr learned my addr 172.21.15.43:0/2170987878 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:17.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.685+0000 7ff8f5d9b700 1 --2- 172.21.15.43:0/2170987878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8f80aec90 0x7ff8f80b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:17.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.685+0000 7ff8f6d9d700 1 --2- 172.21.15.43:0/2170987878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8f80af060 0x7ff8f80b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:17.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.685+0000 7ff8f659c700 1 -- 172.21.15.43:0/2170987878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8f80af060 msgr2=0x7ff8f80b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:17.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.685+0000 7ff8f659c700 1 --2- 172.21.15.43:0/2170987878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8f80af060 0x7ff8f80b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:17.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.685+0000 7ff8f659c700 1 -- 172.21.15.43:0/2170987878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8f80aec90 msgr2=0x7ff8f80b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:17.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.685+0000 7ff8f659c700 1 --2- 172.21.15.43:0/2170987878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8f80aec90 0x7ff8f80b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:17.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.685+0000 7ff8f659c700 1 -- 172.21.15.43:0/2170987878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e00094a0 con 0x7ff8f806ccc0 2023-06-28T20:07:17.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.685+0000 7ff8f5d9b700 1 --2- 172.21.15.43:0/2170987878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8f80aec90 0x7ff8f80b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:17.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.685+0000 7ff8f659c700 1 --2- 172.21.15.43:0/2170987878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8f806ccc0 0x7ff8f8150c00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000ebe0 tx=0x7ff8e0009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:17.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.685+0000 7ff8eeffd700 1 -- 172.21.15.43:0/2170987878 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e000c910 con 0x7ff8f806ccc0 2023-06-28T20:07:17.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.685+0000 7ff8eeffd700 1 -- 172.21.15.43:0/2170987878 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e000ca70 con 0x7ff8f806ccc0 2023-06-28T20:07:17.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.685+0000 7ff8fcb69700 1 -- 172.21.15.43:0/2170987878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e80094a0 con 0x7ff8f806ccc0 2023-06-28T20:07:17.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.685+0000 7ff8eeffd700 1 -- 172.21.15.43:0/2170987878 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e000cde0 con 0x7ff8f806ccc0 2023-06-28T20:07:17.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.685+0000 7ff8fcb69700 1 -- 172.21.15.43:0/2170987878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f80b56a0 con 0x7ff8f806ccc0 2023-06-28T20:07:17.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.686+0000 7ff8fcb69700 1 -- 172.21.15.43:0/2170987878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f80728b0 con 0x7ff8f806ccc0 2023-06-28T20:07:17.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.686+0000 7ff8eeffd700 1 -- 172.21.15.43:0/2170987878 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff8e0013470 con 0x7ff8f806ccc0 2023-06-28T20:07:17.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.687+0000 7ff8eeffd700 1 --2- 172.21.15.43:0/2170987878 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff8d806ace0 0x7ff8d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:17.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.687+0000 7ff8eeffd700 1 -- 172.21.15.43:0/2170987878 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff8e00893e0 con 0x7ff8f806ccc0 2023-06-28T20:07:17.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.687+0000 7ff8f5d9b700 1 --2- 172.21.15.43:0/2170987878 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff8d806ace0 0x7ff8d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:17.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.687+0000 7ff8f5d9b700 1 --2- 172.21.15.43:0/2170987878 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff8d806ace0 0x7ff8d806d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80090b0 tx=0x7ff8e801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:17.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.691+0000 7ff8eeffd700 1 -- 172.21.15.43:0/2170987878 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff8e0054760 con 0x7ff8f806ccc0 2023-06-28T20:07:17.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.887+0000 7ff8fcb69700 1 -- 172.21.15.43:0/2170987878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff8f80c9eb0 con 0x7ff8f806ccc0 2023-06-28T20:07:17.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.888+0000 7ff8eeffd700 1 -- 172.21.15.43:0/2170987878 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff8e0057d80 con 0x7ff8f806ccc0 2023-06-28T20:07:17.891 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-06-28T20:07:17.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.889+0000 7ff8fcb69700 1 -- 172.21.15.43:0/2170987878 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff8d806ace0 msgr2=0x7ff8d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:17.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.889+0000 7ff8fcb69700 1 --2- 172.21.15.43:0/2170987878 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff8d806ace0 0x7ff8d806d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff8e80090b0 tx=0x7ff8e801c040 comp rx=0 tx=0).stop 2023-06-28T20:07:17.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.889+0000 7ff8fcb69700 1 -- 172.21.15.43:0/2170987878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8f806ccc0 msgr2=0x7ff8f8150c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:17.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.889+0000 7ff8fcb69700 1 --2- 172.21.15.43:0/2170987878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8f806ccc0 0x7ff8f8150c00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000ebe0 tx=0x7ff8e0009ec0 comp rx=0 tx=0).stop 2023-06-28T20:07:17.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.889+0000 7ff8fcb69700 1 -- 172.21.15.43:0/2170987878 shutdown_connections 2023-06-28T20:07:17.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.889+0000 7ff8fcb69700 1 --2- 172.21.15.43:0/2170987878 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff8d806ace0 0x7ff8d806d190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:17.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.889+0000 7ff8fcb69700 1 --2- 172.21.15.43:0/2170987878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff8f80af060 0x7ff8f80b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:17.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.889+0000 7ff8fcb69700 1 --2- 172.21.15.43:0/2170987878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8f806ccc0 0x7ff8f8150c00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:17.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.889+0000 7ff8fcb69700 1 --2- 172.21.15.43:0/2170987878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff8f80aec90 0x7ff8f80b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:17.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.889+0000 7ff8fcb69700 1 -- 172.21.15.43:0/2170987878 >> 172.21.15.43:0/2170987878 conn(0x7ff8f806b000 msgr2=0x7ff8f815a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:17.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.890+0000 7ff8fcb69700 1 -- 172.21.15.43:0/2170987878 shutdown_connections 2023-06-28T20:07:17.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:17.890+0000 7ff8fcb69700 1 -- 172.21.15.43:0/2170987878 wait complete. 2023-06-28T20:07:17.905 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-06-28T20:07:18.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.027+0000 7f526da52700 1 Processor -- start 2023-06-28T20:07:18.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.027+0000 7f526da52700 1 -- start start 2023-06-28T20:07:18.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.028+0000 7f526da52700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f526815c4e0 0x7f526806a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:18.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.028+0000 7f526da52700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f526815def0 con 0x7f526806c8b0 2023-06-28T20:07:18.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.028+0000 7f526da52700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52680b5750 con 0x7f526815d680 2023-06-28T20:07:18.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.028+0000 7f526da52700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52680cf710 con 0x7f526815c4e0 2023-06-28T20:07:18.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.028+0000 7f5266ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f526815c4e0 0x7f526806a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:18.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.028+0000 7f52667fc700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f526806c8b0 0x7f52680b4930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41946/0 (socket says 172.21.15.43:41946) 2023-06-28T20:07:18.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.028+0000 7f5266ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f526815c4e0 0x7f526806a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52760/0 (socket says 172.21.15.43:52760) 2023-06-28T20:07:18.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.028+0000 7f52667fc700 1 -- 172.21.15.43:0/650996614 learned_addr learned my addr 172.21.15.43:0/650996614 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:18.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.028+0000 7f5266ffd700 1 -- 172.21.15.43:0/650996614 >> v1:172.21.15.178:6789/0 conn(0x7f526815d680 legacy=0x7f52680b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:18.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.028+0000 7f5266ffd700 1 -- 172.21.15.43:0/650996614 >> v1:172.21.15.43:6789/0 conn(0x7f526806c8b0 legacy=0x7f52680b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.028+0000 7f5266ffd700 1 -- 172.21.15.43:0/650996614 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52680cf150 con 0x7f526815c4e0 2023-06-28T20:07:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.028+0000 7f5265ffb700 1 -- 172.21.15.43:0/650996614 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249296923 0 0) 0x7f526815def0 con 0x7f526806c8b0 2023-06-28T20:07:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.028+0000 7f5266ffd700 1 --2- 172.21.15.43:0/650996614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f526815c4e0 0x7f526806a690 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f525c009840 tx=0x7f525c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51ee02ee7a845af server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.028+0000 7f5265ffb700 1 -- 172.21.15.43:0/650996614 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c004230 con 0x7f526815c4e0 2023-06-28T20:07:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.028+0000 7f5265ffb700 1 -- 172.21.15.43:0/650996614 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525c004390 con 0x7f526815c4e0 2023-06-28T20:07:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.028+0000 7f5265ffb700 1 -- 172.21.15.43:0/650996614 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c0036a0 con 0x7f526815c4e0 2023-06-28T20:07:18.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.029+0000 7f526da52700 1 -- 172.21.15.43:0/650996614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f526815c4e0 msgr2=0x7f526806a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:18.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.029+0000 7f526da52700 1 --2- 172.21.15.43:0/650996614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f526815c4e0 0x7f526806a690 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f525c009840 tx=0x7f525c009b50 comp rx=0 tx=0).stop 2023-06-28T20:07:18.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.029+0000 7f526da52700 1 -- 172.21.15.43:0/650996614 shutdown_connections 2023-06-28T20:07:18.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.029+0000 7f526da52700 1 --2- 172.21.15.43:0/650996614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f526815c4e0 0x7f526806a690 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:18.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.029+0000 7f526da52700 1 -- 172.21.15.43:0/650996614 >> 172.21.15.43:0/650996614 conn(0x7f52680d2eb0 msgr2=0x7f52680d32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:18.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.029+0000 7f526da52700 1 -- 172.21.15.43:0/650996614 shutdown_connections 2023-06-28T20:07:18.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.029+0000 7f526da52700 1 -- 172.21.15.43:0/650996614 wait complete. 2023-06-28T20:07:18.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.030+0000 7f526da52700 1 Processor -- start 2023-06-28T20:07:18.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.030+0000 7f526da52700 1 -- start start 2023-06-28T20:07:18.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.031+0000 7f526da52700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f526806c8b0 0x7f526814ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:18.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.031+0000 7f526da52700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f526815c4e0 0x7f5268150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:18.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.031+0000 7f526da52700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f526815d680 0x7f5268156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:18.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.031+0000 7f526da52700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52680d0ed0 con 0x7f526815d680 2023-06-28T20:07:18.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.031+0000 7f526da52700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52680d2610 con 0x7f526815c4e0 2023-06-28T20:07:18.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.031+0000 7f526da52700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5268151060 con 0x7f526806c8b0 2023-06-28T20:07:18.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.031+0000 7f52677fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f526815d680 0x7f5268156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:18.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.031+0000 7f52677fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f526815d680 0x7f5268156330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52770/0 (socket says 172.21.15.43:52770) 2023-06-28T20:07:18.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.031+0000 7f52677fe700 1 -- 172.21.15.43:0/4252250462 learned_addr learned my addr 172.21.15.43:0/4252250462 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:18.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.031+0000 7f52667fc700 1 --2- 172.21.15.43:0/4252250462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f526815c4e0 0x7f5268150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:18.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.031+0000 7f5266ffd700 1 --2- 172.21.15.43:0/4252250462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f526806c8b0 0x7f526814ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:18.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.031+0000 7f52677fe700 1 -- 172.21.15.43:0/4252250462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f526806c8b0 msgr2=0x7f526814ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:18.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.031+0000 7f52677fe700 1 --2- 172.21.15.43:0/4252250462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f526806c8b0 0x7f526814ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:18.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.031+0000 7f52677fe700 1 -- 172.21.15.43:0/4252250462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f526815c4e0 msgr2=0x7f5268150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:18.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.031+0000 7f52677fe700 1 --2- 172.21.15.43:0/4252250462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f526815c4e0 0x7f5268150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:18.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.032+0000 7f52677fe700 1 -- 172.21.15.43:0/4252250462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525c0094a0 con 0x7f526815d680 2023-06-28T20:07:18.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.032+0000 7f52667fc700 1 --2- 172.21.15.43:0/4252250462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f526815c4e0 0x7f5268150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:18.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.032+0000 7f52677fe700 1 --2- 172.21.15.43:0/4252250462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f526815d680 0x7f5268156330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f526000b9e0 tx=0x7f526000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:18.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.032+0000 7f524ffff700 1 -- 172.21.15.43:0/4252250462 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526000c9b0 con 0x7f526815d680 2023-06-28T20:07:18.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.032+0000 7f524ffff700 1 -- 172.21.15.43:0/4252250462 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526000cb10 con 0x7f526815d680 2023-06-28T20:07:18.039 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.032+0000 7f526da52700 1 -- 172.21.15.43:0/4252250462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5268151340 con 0x7f526815d680 2023-06-28T20:07:18.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.032+0000 7f524ffff700 1 -- 172.21.15.43:0/4252250462 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5260014440 con 0x7f526815d680 2023-06-28T20:07:18.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.032+0000 7f526da52700 1 -- 172.21.15.43:0/4252250462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5268151900 con 0x7f526815d680 2023-06-28T20:07:18.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.033+0000 7f526da52700 1 -- 172.21.15.43:0/4252250462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52680d1080 con 0x7f526815d680 2023-06-28T20:07:18.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.033+0000 7f524ffff700 1 -- 172.21.15.43:0/4252250462 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5260006d50 con 0x7f526815d680 2023-06-28T20:07:18.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.034+0000 7f524ffff700 1 --2- 172.21.15.43:0/4252250462 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f525406ace0 0x7f525406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:18.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.034+0000 7f524ffff700 1 -- 172.21.15.43:0/4252250462 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5260088c40 con 0x7f526815d680 2023-06-28T20:07:18.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.034+0000 7f5266ffd700 1 --2- 172.21.15.43:0/4252250462 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f525406ace0 0x7f525406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:18.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.034+0000 7f5266ffd700 1 --2- 172.21.15.43:0/4252250462 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f525406ace0 0x7f525406d190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f525c009b90 tx=0x7f525c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:18.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.038+0000 7f524ffff700 1 -- 172.21.15.43:0/4252250462 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f52680d1080 con 0x7f526815d680 2023-06-28T20:07:18.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.241+0000 7f526da52700 1 -- 172.21.15.43:0/4252250462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f52680728b0 con 0x7f526815d680 2023-06-28T20:07:18.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.241+0000 7f524ffff700 1 -- 172.21.15.43:0/4252250462 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+126 (secure 0 0 0) 0x7f5260019090 con 0x7f526815d680 2023-06-28T20:07:18.245 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:18.245 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-06-28T20:07:18.245 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:07:18.245 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2023-06-28T20:07:18.245 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": "rbd" 2023-06-28T20:07:18.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:07:18.246 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:07:18.246 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 2, 2023-06-28T20:07:18.246 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2023-06-28T20:07:18.246 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:07:18.246 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-06-28T20:07:18.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.243+0000 7f526da52700 1 -- 172.21.15.43:0/4252250462 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f525406ace0 msgr2=0x7f525406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:18.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.243+0000 7f526da52700 1 --2- 172.21.15.43:0/4252250462 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f525406ace0 0x7f525406d190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f525c009b90 tx=0x7f525c008030 comp rx=0 tx=0).stop 2023-06-28T20:07:18.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.243+0000 7f526da52700 1 -- 172.21.15.43:0/4252250462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f526815d680 msgr2=0x7f5268156330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:18.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.243+0000 7f526da52700 1 --2- 172.21.15.43:0/4252250462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f526815d680 0x7f5268156330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f526000b9e0 tx=0x7f526000bda0 comp rx=0 tx=0).stop 2023-06-28T20:07:18.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.243+0000 7f526da52700 1 -- 172.21.15.43:0/4252250462 shutdown_connections 2023-06-28T20:07:18.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.243+0000 7f526da52700 1 --2- 172.21.15.43:0/4252250462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f526815d680 0x7f5268156330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:18.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.243+0000 7f526da52700 1 --2- 172.21.15.43:0/4252250462 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f525406ace0 0x7f525406d190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:18.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.243+0000 7f526da52700 1 --2- 172.21.15.43:0/4252250462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f526806c8b0 0x7f526814ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:18.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.243+0000 7f526da52700 1 --2- 172.21.15.43:0/4252250462 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f526815c4e0 0x7f5268150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:18.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.243+0000 7f526da52700 1 -- 172.21.15.43:0/4252250462 >> 172.21.15.43:0/4252250462 conn(0x7f52680d2eb0 msgr2=0x7f52680b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:18.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.244+0000 7f526da52700 1 -- 172.21.15.43:0/4252250462 shutdown_connections 2023-06-28T20:07:18.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.244+0000 7f526da52700 1 -- 172.21.15.43:0/4252250462 wait complete. 2023-06-28T20:07:18.259 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-06-28T20:07:18.259 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-06-28T20:07:18.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.385+0000 7fa2c0c6f700 1 Processor -- start 2023-06-28T20:07:18.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.385+0000 7fa2c0c6f700 1 -- start start 2023-06-28T20:07:18.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.386+0000 7fa2c0c6f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa2bc15ceb0 0x7fa2bc15d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:18.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.386+0000 7fa2c0c6f700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa2bc0b4f20 0x7fa2bc0cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:18.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.386+0000 7fa2c0c6f700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2bc0d0e20 0x7fa2bc0d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:18.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.386+0000 7fa2c0c6f700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2bc0b8960 con 0x7fa2bc15ceb0 2023-06-28T20:07:18.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.386+0000 7fa2c0c6f700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2bc0ce770 con 0x7fa2bc0b4f20 2023-06-28T20:07:18.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.386+0000 7fa2c0c6f700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc0cd410 con 0x7fa2bc0d0e20 2023-06-28T20:07:18.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.386+0000 7fa2ba59c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa2bc15ceb0 0x7fa2bc15d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:18.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.386+0000 7fa2ba59c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa2bc15ceb0 0x7fa2bc15d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52774/0 (socket says 172.21.15.43:52774) 2023-06-28T20:07:18.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.386+0000 7fa2ba59c700 1 -- 172.21.15.43:0/4204506432 learned_addr learned my addr 172.21.15.43:0/4204506432 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:18.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.386+0000 7fa2bad9d700 1 --2- 172.21.15.43:0/4204506432 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2bc0d0e20 0x7fa2bc0d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:18.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.386+0000 7fa2b9d9b700 1 --2- 172.21.15.43:0/4204506432 >> v2:172.21.15.178:3300/0 conn(0x7fa2bc0b4f20 0x7fa2bc0cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:18.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.387+0000 7fa2ba59c700 1 -- 172.21.15.43:0/4204506432 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2bc0d0e20 msgr2=0x7fa2bc0d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:18.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.387+0000 7fa2ba59c700 1 --2- 172.21.15.43:0/4204506432 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2bc0d0e20 0x7fa2bc0d12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:18.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.387+0000 7fa2ba59c700 1 -- 172.21.15.43:0/4204506432 >> v2:172.21.15.178:3300/0 conn(0x7fa2bc0b4f20 msgr2=0x7fa2bc0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:18.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.387+0000 7fa2ba59c700 1 --2- 172.21.15.43:0/4204506432 >> v2:172.21.15.178:3300/0 conn(0x7fa2bc0b4f20 0x7fa2bc0cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:18.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.387+0000 7fa2ba59c700 1 -- 172.21.15.43:0/4204506432 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2bc0d0990 con 0x7fa2bc15ceb0 2023-06-28T20:07:18.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.387+0000 7fa2ba59c700 1 --2- 172.21.15.43:0/4204506432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2bc15ceb0 0x7fa2bc15d280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0cd3a0 tx=0x7fa2b0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8472ff0d21f4af1 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:18.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.387+0000 7fa2b959a700 1 -- 172.21.15.43:0/4204506432 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0011830 con 0x7fa2bc15ceb0 2023-06-28T20:07:18.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.387+0000 7fa2b959a700 1 -- 172.21.15.43:0/4204506432 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b0011990 con 0x7fa2bc15ceb0 2023-06-28T20:07:18.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.387+0000 7fa2b959a700 1 -- 172.21.15.43:0/4204506432 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b0011d20 con 0x7fa2bc15ceb0 2023-06-28T20:07:18.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.387+0000 7fa2c0c6f700 1 -- 172.21.15.43:0/4204506432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2bc15ceb0 msgr2=0x7fa2bc15d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:18.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.387+0000 7fa2c0c6f700 1 --2- 172.21.15.43:0/4204506432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2bc15ceb0 0x7fa2bc15d280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc0cd3a0 tx=0x7fa2b0009bd0 comp rx=0 tx=0).stop 2023-06-28T20:07:18.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.387+0000 7fa2c0c6f700 1 -- 172.21.15.43:0/4204506432 shutdown_connections 2023-06-28T20:07:18.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.387+0000 7fa2c0c6f700 1 --2- 172.21.15.43:0/4204506432 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2bc0d0e20 0x7fa2bc0d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:18.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.387+0000 7fa2c0c6f700 1 --2- 172.21.15.43:0/4204506432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2bc15ceb0 0x7fa2bc15d280 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:18.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.387+0000 7fa2c0c6f700 1 --2- 172.21.15.43:0/4204506432 >> v2:172.21.15.178:3300/0 conn(0x7fa2bc0b4f20 0x7fa2bc0cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:18.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.387+0000 7fa2c0c6f700 1 -- 172.21.15.43:0/4204506432 >> 172.21.15.43:0/4204506432 conn(0x7fa2bc06a5c0 msgr2=0x7fa2bc06a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:18.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.388+0000 7fa2c0c6f700 1 -- 172.21.15.43:0/4204506432 shutdown_connections 2023-06-28T20:07:18.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.388+0000 7fa2c0c6f700 1 -- 172.21.15.43:0/4204506432 wait complete. 2023-06-28T20:07:18.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.389+0000 7fa2c0c6f700 1 Processor -- start 2023-06-28T20:07:18.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.389+0000 7fa2c0c6f700 1 -- start start 2023-06-28T20:07:18.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.390+0000 7fa2c0c6f700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2bc0b4f20 0x7fa2bc157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:18.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.390+0000 7fa2c0c6f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2bc0d0e20 0x7fa2bc156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:18.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.390+0000 7fa2c0c6f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2bc15ceb0 0x7fa2bc0cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:18.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.390+0000 7fa2c0c6f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc0c6b60 con 0x7fa2bc15ceb0 2023-06-28T20:07:18.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.390+0000 7fa2c0c6f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc157970 con 0x7fa2bc0d0e20 2023-06-28T20:07:18.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.390+0000 7fa2c0c6f700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2bc157c70 con 0x7fa2bc0b4f20 2023-06-28T20:07:18.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.390+0000 7fa2bad9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2bc15ceb0 0x7fa2bc0cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:18.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.390+0000 7fa2bad9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2bc15ceb0 0x7fa2bc0cb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52784/0 (socket says 172.21.15.43:52784) 2023-06-28T20:07:18.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.390+0000 7fa2bad9d700 1 -- 172.21.15.43:0/2004058183 learned_addr learned my addr 172.21.15.43:0/2004058183 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:18.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.391+0000 7fa2ba59c700 1 --2- 172.21.15.43:0/2004058183 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2bc0b4f20 0x7fa2bc157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:18.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.391+0000 7fa2b9d9b700 1 --2- 172.21.15.43:0/2004058183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2bc0d0e20 0x7fa2bc156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:18.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.391+0000 7fa2bad9d700 1 -- 172.21.15.43:0/2004058183 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2bc0b4f20 msgr2=0x7fa2bc157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:18.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.391+0000 7fa2bad9d700 1 --2- 172.21.15.43:0/2004058183 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2bc0b4f20 0x7fa2bc157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:18.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.391+0000 7fa2bad9d700 1 -- 172.21.15.43:0/2004058183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2bc0d0e20 msgr2=0x7fa2bc156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:18.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.391+0000 7fa2bad9d700 1 --2- 172.21.15.43:0/2004058183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2bc0d0e20 0x7fa2bc156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:18.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.391+0000 7fa2bad9d700 1 -- 172.21.15.43:0/2004058183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b0009550 con 0x7fa2bc15ceb0 2023-06-28T20:07:18.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.391+0000 7fa2bad9d700 1 --2- 172.21.15.43:0/2004058183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2bc15ceb0 0x7fa2bc0cb080 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400efc0 tx=0x7fa2b400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:18.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.391+0000 7fa2aaffd700 1 -- 172.21.15.43:0/2004058183 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4009ea0 con 0x7fa2bc15ceb0 2023-06-28T20:07:18.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.391+0000 7fa2aaffd700 1 -- 172.21.15.43:0/2004058183 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2b400f040 con 0x7fa2bc15ceb0 2023-06-28T20:07:18.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.391+0000 7fa2aaffd700 1 -- 172.21.15.43:0/2004058183 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2b400ced0 con 0x7fa2bc15ceb0 2023-06-28T20:07:18.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.391+0000 7fa2c0c6f700 1 -- 172.21.15.43:0/2004058183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2bc157f50 con 0x7fa2bc15ceb0 2023-06-28T20:07:18.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.391+0000 7fa2c0c6f700 1 -- 172.21.15.43:0/2004058183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2bc158510 con 0x7fa2bc15ceb0 2023-06-28T20:07:18.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.392+0000 7fa2c0c6f700 1 -- 172.21.15.43:0/2004058183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2bc158730 con 0x7fa2bc15ceb0 2023-06-28T20:07:18.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.393+0000 7fa2aaffd700 1 -- 172.21.15.43:0/2004058183 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa2b4014480 con 0x7fa2bc15ceb0 2023-06-28T20:07:18.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.393+0000 7fa2aaffd700 1 --2- 172.21.15.43:0/2004058183 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa29c06ace0 0x7fa29c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:18.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.393+0000 7fa2aaffd700 1 -- 172.21.15.43:0/2004058183 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa2b4089c20 con 0x7fa2bc15ceb0 2023-06-28T20:07:18.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.393+0000 7fa2ba59c700 1 --2- 172.21.15.43:0/2004058183 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa29c06ace0 0x7fa29c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:18.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.394+0000 7fa2ba59c700 1 --2- 172.21.15.43:0/2004058183 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa29c06ace0 0x7fa29c06d190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0014040 tx=0x7fa2b0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:18.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.397+0000 7fa2aaffd700 1 -- 172.21.15.43:0/2004058183 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa2b4055020 con 0x7fa2bc15ceb0 2023-06-28T20:07:18.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:18.594+0000 7fa2c0c6f700 1 -- 172.21.15.43:0/2004058183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa2bc0728b0 con 0x7fa2bc15ceb0 2023-06-28T20:07:19.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.355+0000 7fa2aaffd700 1 -- 172.21.15.43:0/2004058183 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa2b4058640 con 0x7fa2bc15ceb0 2023-06-28T20:07:19.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.360+0000 7fa2c0c6f700 1 -- 172.21.15.43:0/2004058183 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa29c06ace0 msgr2=0x7fa29c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:19.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.360+0000 7fa2c0c6f700 1 --2- 172.21.15.43:0/2004058183 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa29c06ace0 0x7fa29c06d190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0014040 tx=0x7fa2b0008030 comp rx=0 tx=0).stop 2023-06-28T20:07:19.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.360+0000 7fa2c0c6f700 1 -- 172.21.15.43:0/2004058183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2bc15ceb0 msgr2=0x7fa2bc0cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:19.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.360+0000 7fa2c0c6f700 1 --2- 172.21.15.43:0/2004058183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2bc15ceb0 0x7fa2bc0cb080 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa2b400efc0 tx=0x7fa2b400c3a0 comp rx=0 tx=0).stop 2023-06-28T20:07:19.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.361+0000 7fa2c0c6f700 1 -- 172.21.15.43:0/2004058183 shutdown_connections 2023-06-28T20:07:19.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.361+0000 7fa2c0c6f700 1 --2- 172.21.15.43:0/2004058183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa2bc15ceb0 0x7fa2bc0cb080 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:19.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.361+0000 7fa2c0c6f700 1 --2- 172.21.15.43:0/2004058183 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa29c06ace0 0x7fa29c06d190 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:19.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.361+0000 7fa2c0c6f700 1 --2- 172.21.15.43:0/2004058183 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa2bc0b4f20 0x7fa2bc157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:19.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.361+0000 7fa2c0c6f700 1 --2- 172.21.15.43:0/2004058183 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2bc0d0e20 0x7fa2bc156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:19.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.361+0000 7fa2c0c6f700 1 -- 172.21.15.43:0/2004058183 >> 172.21.15.43:0/2004058183 conn(0x7fa2bc06a5c0 msgr2=0x7fa2bc0b5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:19.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.361+0000 7fa2c0c6f700 1 -- 172.21.15.43:0/2004058183 shutdown_connections 2023-06-28T20:07:19.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.362+0000 7fa2c0c6f700 1 -- 172.21.15.43:0/2004058183 wait complete. 2023-06-28T20:07:19.366 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2023-06-28T20:07:19.380 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-06-28T20:07:19.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.517+0000 7f698d99f700 1 Processor -- start 2023-06-28T20:07:19.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.517+0000 7f698d99f700 1 -- start start 2023-06-28T20:07:19.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.518+0000 7f698d99f700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f698815c2a0 0x7f698806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:19.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.518+0000 7f698d99f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f698815c670 0x7f698806d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:19.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.518+0000 7f698d99f700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f698815d4e0 con 0x7f698806c320 2023-06-28T20:07:19.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.518+0000 7f698d99f700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f698806dd60 con 0x7f698815c670 2023-06-28T20:07:19.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.518+0000 7f698d99f700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f698815cbb0 con 0x7f698815c2a0 2023-06-28T20:07:19.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.518+0000 7f6987fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f698815c670 0x7f698806d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:19.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.518+0000 7f6987fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f698815c670 0x7f698806d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52790/0 (socket says 172.21.15.43:52790) 2023-06-28T20:07:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.518+0000 7f6987fff700 1 -- 172.21.15.43:0/1722547465 learned_addr learned my addr 172.21.15.43:0/1722547465 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.518+0000 7f698c99d700 1 --2- 172.21.15.43:0/1722547465 >> v2:172.21.15.178:3300/0 conn(0x7f698815c2a0 0x7f698806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.518+0000 7f6987fff700 1 -- 172.21.15.43:0/1722547465 >> v2:172.21.15.178:3300/0 conn(0x7f698815c2a0 msgr2=0x7f698806d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.518+0000 7f6987fff700 1 --2- 172.21.15.43:0/1722547465 >> v2:172.21.15.178:3300/0 conn(0x7f698815c2a0 0x7f698806d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.518+0000 7f6987fff700 1 -- 172.21.15.43:0/1722547465 >> v1:172.21.15.178:6789/0 conn(0x7f698806c320 legacy=0x7f698806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.518+0000 7f6987fff700 1 -- 172.21.15.43:0/1722547465 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698815ce90 con 0x7f698815c670 2023-06-28T20:07:19.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.519+0000 7f6987fff700 1 --2- 172.21.15.43:0/1722547465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f698815c670 0x7f698806d630 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f698806e340 tx=0x7f6974009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b904af22e4a03b server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.519+0000 7f69877fe700 1 -- 172.21.15.43:0/1722547465 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974004230 con 0x7f698815c670 2023-06-28T20:07:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.519+0000 7f69877fe700 1 -- 172.21.15.43:0/1722547465 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6974004390 con 0x7f698815c670 2023-06-28T20:07:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.519+0000 7f69877fe700 1 -- 172.21.15.43:0/1722547465 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69740036a0 con 0x7f698815c670 2023-06-28T20:07:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.519+0000 7f698d99f700 1 -- 172.21.15.43:0/1722547465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f698815c670 msgr2=0x7f698806d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.519+0000 7f698d99f700 1 --2- 172.21.15.43:0/1722547465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f698815c670 0x7f698806d630 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f698806e340 tx=0x7f6974009bd0 comp rx=0 tx=0).stop 2023-06-28T20:07:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.519+0000 7f698d99f700 1 -- 172.21.15.43:0/1722547465 shutdown_connections 2023-06-28T20:07:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.519+0000 7f698d99f700 1 --2- 172.21.15.43:0/1722547465 >> v2:172.21.15.178:3300/0 conn(0x7f698815c2a0 0x7f698806d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:19.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.519+0000 7f698d99f700 1 --2- 172.21.15.43:0/1722547465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f698815c670 0x7f698806d630 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.519+0000 7f698d99f700 1 -- 172.21.15.43:0/1722547465 >> 172.21.15.43:0/1722547465 conn(0x7f698806bc90 msgr2=0x7f69880cc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.519+0000 7f698d99f700 1 -- 172.21.15.43:0/1722547465 shutdown_connections 2023-06-28T20:07:19.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.519+0000 7f698d99f700 1 -- 172.21.15.43:0/1722547465 wait complete. 2023-06-28T20:07:19.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.521+0000 7f698d99f700 1 Processor -- start 2023-06-28T20:07:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.521+0000 7f698d99f700 1 -- start start 2023-06-28T20:07:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.522+0000 7f698d99f700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698806c320 0x7f69880c3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.522+0000 7f698d99f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f698815c2a0 0x7f69880b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.522+0000 7f698d99f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f698815c670 0x7f6988155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.522+0000 7f698d99f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69880c3f80 con 0x7f698815c2a0 2023-06-28T20:07:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.522+0000 7f698d99f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69880b1150 con 0x7f698815c670 2023-06-28T20:07:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.522+0000 7f698d99f700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69880b0e20 con 0x7f698806c320 2023-06-28T20:07:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.522+0000 7f6987fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f698815c2a0 0x7f69880b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.522+0000 7f6987fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f698815c2a0 0x7f69880b15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52796/0 (socket says 172.21.15.43:52796) 2023-06-28T20:07:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.522+0000 7f6987fff700 1 -- 172.21.15.43:0/4105372157 learned_addr learned my addr 172.21.15.43:0/4105372157 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.522+0000 7f698d19e700 1 --2- 172.21.15.43:0/4105372157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f698815c670 0x7f6988155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.522+0000 7f698c99d700 1 --2- 172.21.15.43:0/4105372157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698806c320 0x7f69880c3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:19.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.522+0000 7f6987fff700 1 -- 172.21.15.43:0/4105372157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698806c320 msgr2=0x7f69880c3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.522+0000 7f6987fff700 1 --2- 172.21.15.43:0/4105372157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698806c320 0x7f69880c3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.522+0000 7f6987fff700 1 -- 172.21.15.43:0/4105372157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f698815c670 msgr2=0x7f6988155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.522+0000 7f6987fff700 1 --2- 172.21.15.43:0/4105372157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f698815c670 0x7f6988155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.522+0000 7f6987fff700 1 -- 172.21.15.43:0/4105372157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6974009550 con 0x7f698815c2a0 2023-06-28T20:07:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.522+0000 7f6987fff700 1 --2- 172.21.15.43:0/4105372157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f698815c2a0 0x7f69880b15e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6974004000 tx=0x7f6974004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.522+0000 7f69857fa700 1 -- 172.21.15.43:0/4105372157 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974003ea0 con 0x7f698815c2a0 2023-06-28T20:07:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.522+0000 7f69857fa700 1 -- 172.21.15.43:0/4105372157 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6974011630 con 0x7f698815c2a0 2023-06-28T20:07:19.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.522+0000 7f69857fa700 1 -- 172.21.15.43:0/4105372157 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69740119e0 con 0x7f698815c2a0 2023-06-28T20:07:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.522+0000 7f698d99f700 1 -- 172.21.15.43:0/4105372157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69880c30b0 con 0x7f698815c2a0 2023-06-28T20:07:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.522+0000 7f698d99f700 1 -- 172.21.15.43:0/4105372157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69880c3670 con 0x7f698815c2a0 2023-06-28T20:07:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.524+0000 7f698d99f700 1 -- 172.21.15.43:0/4105372157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f698804ea20 con 0x7f698815c2a0 2023-06-28T20:07:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.524+0000 7f69857fa700 1 -- 172.21.15.43:0/4105372157 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6974011b40 con 0x7f698815c2a0 2023-06-28T20:07:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.524+0000 7f69857fa700 1 --2- 172.21.15.43:0/4105372157 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f696c06ace0 0x7f696c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.524+0000 7f69857fa700 1 -- 172.21.15.43:0/4105372157 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f697408a790 con 0x7f698815c2a0 2023-06-28T20:07:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.524+0000 7f698c99d700 1 --2- 172.21.15.43:0/4105372157 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f696c06ace0 0x7f696c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:19.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.525+0000 7f698c99d700 1 --2- 172.21.15.43:0/4105372157 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f696c06ace0 0x7f696c06d190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f697c00ab60 tx=0x7f697c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:19.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.529+0000 7f69857fa700 1 -- 172.21.15.43:0/4105372157 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6974055ab0 con 0x7f698815c2a0 2023-06-28T20:07:19.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:19.729+0000 7f698d99f700 1 -- 172.21.15.43:0/4105372157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f69880b1290 con 0x7f698815c2a0 2023-06-28T20:07:20.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.363+0000 7f69857fa700 1 -- 172.21.15.43:0/4105372157 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7f69740590d0 con 0x7f698815c2a0 2023-06-28T20:07:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.370+0000 7f698d99f700 1 -- 172.21.15.43:0/4105372157 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f696c06ace0 msgr2=0x7f696c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.370+0000 7f698d99f700 1 --2- 172.21.15.43:0/4105372157 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f696c06ace0 0x7f696c06d190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f697c00ab60 tx=0x7f697c008040 comp rx=0 tx=0).stop 2023-06-28T20:07:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.370+0000 7f698d99f700 1 -- 172.21.15.43:0/4105372157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f698815c2a0 msgr2=0x7f69880b15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.370+0000 7f698d99f700 1 --2- 172.21.15.43:0/4105372157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f698815c2a0 0x7f69880b15e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6974004000 tx=0x7f6974004750 comp rx=0 tx=0).stop 2023-06-28T20:07:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.370+0000 7f698d99f700 1 -- 172.21.15.43:0/4105372157 shutdown_connections 2023-06-28T20:07:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.370+0000 7f698d99f700 1 --2- 172.21.15.43:0/4105372157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f698815c670 0x7f6988155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.370+0000 7f698d99f700 1 --2- 172.21.15.43:0/4105372157 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f696c06ace0 0x7f696c06d190 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.370+0000 7f698d99f700 1 --2- 172.21.15.43:0/4105372157 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f698806c320 0x7f69880c3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.370+0000 7f698d99f700 1 --2- 172.21.15.43:0/4105372157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f698815c2a0 0x7f69880b15e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.370+0000 7f698d99f700 1 -- 172.21.15.43:0/4105372157 >> 172.21.15.43:0/4105372157 conn(0x7f698806bc90 msgr2=0x7f69880add60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:20.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.371+0000 7f698d99f700 1 -- 172.21.15.43:0/4105372157 shutdown_connections 2023-06-28T20:07:20.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.372+0000 7f698d99f700 1 -- 172.21.15.43:0/4105372157 wait complete. 2023-06-28T20:07:20.385 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2023-06-28T20:07:20.399 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-06-28T20:07:20.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.530+0000 7efdab04f700 1 Processor -- start 2023-06-28T20:07:20.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.530+0000 7efdab04f700 1 -- start start 2023-06-28T20:07:20.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.531+0000 7efdab04f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7efda40af060 0x7efda4160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.531+0000 7efdab04f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda4160250 con 0x7efda40aec90 2023-06-28T20:07:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.531+0000 7efdab04f700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda4071410 con 0x7efda406ccc0 2023-06-28T20:07:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.531+0000 7efdab04f700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda415cb30 con 0x7efda40af060 2023-06-28T20:07:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.531+0000 7efda3fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7efda40af060 0x7efda4160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.531+0000 7efda8deb700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7efda40aec90 0x7efda4070bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41988/0 (socket says 172.21.15.43:41988) 2023-06-28T20:07:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.531+0000 7efda3fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7efda40af060 0x7efda4160cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52804/0 (socket says 172.21.15.43:52804) 2023-06-28T20:07:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.531+0000 7efda8deb700 1 -- 172.21.15.43:0/1513621354 learned_addr learned my addr 172.21.15.43:0/1513621354 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:20.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.531+0000 7efda3fff700 1 -- 172.21.15.43:0/1513621354 >> v1:172.21.15.178:6789/0 conn(0x7efda406ccc0 legacy=0x7efda40b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:20.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.531+0000 7efda3fff700 1 -- 172.21.15.43:0/1513621354 >> v1:172.21.15.43:6789/0 conn(0x7efda40aec90 legacy=0x7efda4070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:20.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.531+0000 7efda3fff700 1 -- 172.21.15.43:0/1513621354 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda41604f0 con 0x7efda40af060 2023-06-28T20:07:20.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.531+0000 7efda37fe700 1 -- 172.21.15.43:0/1513621354 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2735058843 0 0) 0x7efda4160250 con 0x7efda40aec90 2023-06-28T20:07:20.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.532+0000 7efda3fff700 1 --2- 172.21.15.43:0/1513621354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efda40af060 0x7efda4160cf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efd90009840 tx=0x7efd90009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb2d3b9c40a224d server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:20.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.532+0000 7efda37fe700 1 -- 172.21.15.43:0/1513621354 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd90011830 con 0x7efda40af060 2023-06-28T20:07:20.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.532+0000 7efda37fe700 1 -- 172.21.15.43:0/1513621354 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd90011990 con 0x7efda40af060 2023-06-28T20:07:20.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.532+0000 7efda37fe700 1 -- 172.21.15.43:0/1513621354 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd90011d20 con 0x7efda40af060 2023-06-28T20:07:20.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.532+0000 7efdab04f700 1 -- 172.21.15.43:0/1513621354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efda40af060 msgr2=0x7efda4160cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:20.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.532+0000 7efdab04f700 1 --2- 172.21.15.43:0/1513621354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efda40af060 0x7efda4160cf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efd90009840 tx=0x7efd90009b50 comp rx=0 tx=0).stop 2023-06-28T20:07:20.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.532+0000 7efdab04f700 1 -- 172.21.15.43:0/1513621354 shutdown_connections 2023-06-28T20:07:20.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.532+0000 7efdab04f700 1 --2- 172.21.15.43:0/1513621354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efda40af060 0x7efda4160cf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:20.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.532+0000 7efdab04f700 1 -- 172.21.15.43:0/1513621354 >> 172.21.15.43:0/1513621354 conn(0x7efda406b000 msgr2=0x7efda406b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:20.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.532+0000 7efdab04f700 1 -- 172.21.15.43:0/1513621354 shutdown_connections 2023-06-28T20:07:20.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.532+0000 7efdab04f700 1 -- 172.21.15.43:0/1513621354 wait complete. 2023-06-28T20:07:20.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.534+0000 7efdab04f700 1 Processor -- start 2023-06-28T20:07:20.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.534+0000 7efdab04f700 1 -- start start 2023-06-28T20:07:20.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.534+0000 7efdab04f700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efda406ccc0 0x7efda40b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:20.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.534+0000 7efdab04f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efda40aec90 0x7efda4159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:20.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.535+0000 7efdab04f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efda40af060 0x7efda4153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:20.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.535+0000 7efdab04f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda40b3c80 con 0x7efda40af060 2023-06-28T20:07:20.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.535+0000 7efdab04f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda40c8a70 con 0x7efda40aec90 2023-06-28T20:07:20.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.535+0000 7efdab04f700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda41541e0 con 0x7efda406ccc0 2023-06-28T20:07:20.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.535+0000 7efda95ec700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efda40af060 0x7efda4153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:20.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.535+0000 7efda95ec700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efda40af060 0x7efda4153ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52812/0 (socket says 172.21.15.43:52812) 2023-06-28T20:07:20.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.535+0000 7efda95ec700 1 -- 172.21.15.43:0/2918949554 learned_addr learned my addr 172.21.15.43:0/2918949554 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.535+0000 7efda3fff700 1 --2- 172.21.15.43:0/2918949554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efda40aec90 0x7efda4159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.535+0000 7efda8deb700 1 --2- 172.21.15.43:0/2918949554 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efda406ccc0 0x7efda40b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.535+0000 7efda95ec700 1 -- 172.21.15.43:0/2918949554 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efda406ccc0 msgr2=0x7efda40b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.535+0000 7efda95ec700 1 --2- 172.21.15.43:0/2918949554 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efda406ccc0 0x7efda40b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.535+0000 7efda95ec700 1 -- 172.21.15.43:0/2918949554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efda40aec90 msgr2=0x7efda4159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.535+0000 7efda95ec700 1 --2- 172.21.15.43:0/2918949554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efda40aec90 0x7efda4159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.535+0000 7efda95ec700 1 -- 172.21.15.43:0/2918949554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd900094a0 con 0x7efda40af060 2023-06-28T20:07:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.535+0000 7efda3fff700 1 --2- 172.21.15.43:0/2918949554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efda40aec90 0x7efda4159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.535+0000 7efda8deb700 1 --2- 172.21.15.43:0/2918949554 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efda406ccc0 0x7efda40b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.535+0000 7efda95ec700 1 --2- 172.21.15.43:0/2918949554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efda40af060 0x7efda4153ca0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00cc50 tx=0x7efd9c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.535+0000 7efda17fa700 1 -- 172.21.15.43:0/2918949554 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c00ad80 con 0x7efda40af060 2023-06-28T20:07:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.535+0000 7efda17fa700 1 -- 172.21.15.43:0/2918949554 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9c00ce50 con 0x7efda40af060 2023-06-28T20:07:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.535+0000 7efdab04f700 1 -- 172.21.15.43:0/2918949554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda40b4aa0 con 0x7efda40af060 2023-06-28T20:07:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.535+0000 7efda17fa700 1 -- 172.21.15.43:0/2918949554 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c003c40 con 0x7efda40af060 2023-06-28T20:07:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.535+0000 7efdab04f700 1 -- 172.21.15.43:0/2918949554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda40c8fe0 con 0x7efda40af060 2023-06-28T20:07:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.536+0000 7efdab04f700 1 -- 172.21.15.43:0/2918949554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda40728b0 con 0x7efda40af060 2023-06-28T20:07:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.537+0000 7efda17fa700 1 -- 172.21.15.43:0/2918949554 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efd9c0050d0 con 0x7efda40af060 2023-06-28T20:07:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.537+0000 7efda17fa700 1 --2- 172.21.15.43:0/2918949554 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efd8806ace0 0x7efd8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.537+0000 7efda17fa700 1 -- 172.21.15.43:0/2918949554 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5962+0+0 (secure 0 0 0) 0x7efd9c089530 con 0x7efda40af060 2023-06-28T20:07:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.537+0000 7efda8deb700 1 --2- 172.21.15.43:0/2918949554 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efd8806ace0 0x7efd8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.538+0000 7efda8deb700 1 --2- 172.21.15.43:0/2918949554 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efd8806ace0 0x7efd8806d190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efd98000f60 tx=0x7efd98009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.542+0000 7efda17fa700 1 -- 172.21.15.43:0/2918949554 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efd9c054640 con 0x7efda40af060 2023-06-28T20:07:20.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:20.787+0000 7efdab04f700 1 -- 172.21.15.43:0/2918949554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7efda40cfd60 con 0x7efda40af060 2023-06-28T20:07:21.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.365+0000 7efda17fa700 1 -- 172.21.15.43:0/2918949554 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7efd9c057c60 con 0x7efda40af060 2023-06-28T20:07:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.380+0000 7efdab04f700 1 -- 172.21.15.43:0/2918949554 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efd8806ace0 msgr2=0x7efd8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:21.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.380+0000 7efdab04f700 1 --2- 172.21.15.43:0/2918949554 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efd8806ace0 0x7efd8806d190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efd98000f60 tx=0x7efd98009040 comp rx=0 tx=0).stop 2023-06-28T20:07:21.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.380+0000 7efdab04f700 1 -- 172.21.15.43:0/2918949554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efda40af060 msgr2=0x7efda4153ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:21.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.380+0000 7efdab04f700 1 --2- 172.21.15.43:0/2918949554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efda40af060 0x7efda4153ca0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00cc50 tx=0x7efd9c00a340 comp rx=0 tx=0).stop 2023-06-28T20:07:21.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.381+0000 7efdab04f700 1 -- 172.21.15.43:0/2918949554 shutdown_connections 2023-06-28T20:07:21.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.381+0000 7efdab04f700 1 --2- 172.21.15.43:0/2918949554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efda40af060 0x7efda4153ca0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:21.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.381+0000 7efdab04f700 1 --2- 172.21.15.43:0/2918949554 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efd8806ace0 0x7efd8806d190 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:21.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.381+0000 7efdab04f700 1 --2- 172.21.15.43:0/2918949554 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efda406ccc0 0x7efda40b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:21.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.381+0000 7efdab04f700 1 --2- 172.21.15.43:0/2918949554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efda40aec90 0x7efda4159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:21.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.381+0000 7efdab04f700 1 -- 172.21.15.43:0/2918949554 >> 172.21.15.43:0/2918949554 conn(0x7efda406b000 msgr2=0x7efda40b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:21.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.383+0000 7efdab04f700 1 -- 172.21.15.43:0/2918949554 shutdown_connections 2023-06-28T20:07:21.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.383+0000 7efdab04f700 1 -- 172.21.15.43:0/2918949554 wait complete. 2023-06-28T20:07:21.397 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 3 and data pool 4 2023-06-28T20:07:21.412 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:07:21.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.541+0000 7f529a2a8700 1 Processor -- start 2023-06-28T20:07:21.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.542+0000 7f529a2a8700 1 -- start start 2023-06-28T20:07:21.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.542+0000 7f529a2a8700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f52940d10f0 0x7f52940ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:21.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.542+0000 7f529a2a8700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f52940cedf0 0x7f52940cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:21.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.542+0000 7f529a2a8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52940cb8e0 0x7f52940cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:21.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.542+0000 7f529a2a8700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52940cf260 con 0x7f52940d10f0 2023-06-28T20:07:21.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.542+0000 7f529a2a8700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529415ab50 con 0x7f52940cedf0 2023-06-28T20:07:21.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.542+0000 7f529a2a8700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52940cd1b0 con 0x7f52940cb8e0 2023-06-28T20:07:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.542+0000 7f5293fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f52940d10f0 0x7f52940ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.543+0000 7f5293fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f52940d10f0 0x7f52940ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52828/0 (socket says 172.21.15.43:52828) 2023-06-28T20:07:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.543+0000 7f5293fff700 1 -- 172.21.15.43:0/1236150340 learned_addr learned my addr 172.21.15.43:0/1236150340 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.543+0000 7f5298845700 1 --2- 172.21.15.43:0/1236150340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52940cb8e0 0x7f52940cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.543+0000 7f52937fe700 1 --2- 172.21.15.43:0/1236150340 >> v2:172.21.15.178:3300/0 conn(0x7f52940cedf0 0x7f52940cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:21.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.543+0000 7f5293fff700 1 -- 172.21.15.43:0/1236150340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52940cb8e0 msgr2=0x7f52940cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:21.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.543+0000 7f5293fff700 1 --2- 172.21.15.43:0/1236150340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52940cb8e0 0x7f52940cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:21.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.543+0000 7f5293fff700 1 -- 172.21.15.43:0/1236150340 >> v2:172.21.15.178:3300/0 conn(0x7f52940cedf0 msgr2=0x7f52940cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:21.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.543+0000 7f5293fff700 1 --2- 172.21.15.43:0/1236150340 >> v2:172.21.15.178:3300/0 conn(0x7f52940cedf0 0x7f52940cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:21.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.543+0000 7f5293fff700 1 -- 172.21.15.43:0/1236150340 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52940cc110 con 0x7f52940d10f0 2023-06-28T20:07:21.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.543+0000 7f5293fff700 1 --2- 172.21.15.43:0/1236150340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52940d10f0 0x7f52940ce8b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f529404e4a0 tx=0x7f5284009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb23e789581f24 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:21.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.543+0000 7f5292ffd700 1 -- 172.21.15.43:0/1236150340 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284005e30 con 0x7f52940d10f0 2023-06-28T20:07:21.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.543+0000 7f5292ffd700 1 -- 172.21.15.43:0/1236150340 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5284004030 con 0x7f52940d10f0 2023-06-28T20:07:21.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.543+0000 7f5292ffd700 1 -- 172.21.15.43:0/1236150340 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52840043e0 con 0x7f52940d10f0 2023-06-28T20:07:21.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.544+0000 7f529a2a8700 1 -- 172.21.15.43:0/1236150340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52940d10f0 msgr2=0x7f52940ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:21.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.544+0000 7f529a2a8700 1 --2- 172.21.15.43:0/1236150340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52940d10f0 0x7f52940ce8b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f529404e4a0 tx=0x7f5284009bd0 comp rx=0 tx=0).stop 2023-06-28T20:07:21.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.544+0000 7f529a2a8700 1 -- 172.21.15.43:0/1236150340 shutdown_connections 2023-06-28T20:07:21.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.544+0000 7f529a2a8700 1 --2- 172.21.15.43:0/1236150340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52940cb8e0 0x7f52940cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:21.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.544+0000 7f529a2a8700 1 --2- 172.21.15.43:0/1236150340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52940d10f0 0x7f52940ce8b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:21.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.544+0000 7f529a2a8700 1 --2- 172.21.15.43:0/1236150340 >> v2:172.21.15.178:3300/0 conn(0x7f52940cedf0 0x7f52940cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:21.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.544+0000 7f529a2a8700 1 -- 172.21.15.43:0/1236150340 >> 172.21.15.43:0/1236150340 conn(0x7f52940664b0 msgr2=0x7f529409a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:21.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.544+0000 7f529a2a8700 1 -- 172.21.15.43:0/1236150340 shutdown_connections 2023-06-28T20:07:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.544+0000 7f529a2a8700 1 -- 172.21.15.43:0/1236150340 wait complete. 2023-06-28T20:07:21.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.545+0000 7f529a2a8700 1 Processor -- start 2023-06-28T20:07:21.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.545+0000 7f529a2a8700 1 -- start start 2023-06-28T20:07:21.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.546+0000 7f529a2a8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52940cb8e0 0x7f52940b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:21.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.546+0000 7f529a2a8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52940cedf0 0x7f5294159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:21.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.546+0000 7f529a2a8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52940d10f0 0x7f529409bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:21.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.546+0000 7f529a2a8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52940b5020 con 0x7f52940d10f0 2023-06-28T20:07:21.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.546+0000 7f529a2a8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52941f0d90 con 0x7f52940cb8e0 2023-06-28T20:07:21.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.546+0000 7f529a2a8700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52940b43a0 con 0x7f52940cedf0 2023-06-28T20:07:21.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.546+0000 7f5298845700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52940d10f0 0x7f529409bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:21.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.546+0000 7f5298845700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52940d10f0 0x7f529409bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52838/0 (socket says 172.21.15.43:52838) 2023-06-28T20:07:21.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.546+0000 7f5298845700 1 -- 172.21.15.43:0/8756410 learned_addr learned my addr 172.21.15.43:0/8756410 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:21.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.546+0000 7f5293fff700 1 --2- 172.21.15.43:0/8756410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52940cb8e0 0x7f52940b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:21.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.546+0000 7f52937fe700 1 --2- 172.21.15.43:0/8756410 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52940cedf0 0x7f5294159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:21.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.546+0000 7f5298845700 1 -- 172.21.15.43:0/8756410 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52940cedf0 msgr2=0x7f5294159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.546+0000 7f5298845700 1 --2- 172.21.15.43:0/8756410 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52940cedf0 0x7f5294159b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.546+0000 7f5298845700 1 -- 172.21.15.43:0/8756410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52940cb8e0 msgr2=0x7f52940b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.546+0000 7f5298845700 1 --2- 172.21.15.43:0/8756410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52940cb8e0 0x7f52940b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.546+0000 7f5298845700 1 -- 172.21.15.43:0/8756410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5284009500 con 0x7f52940d10f0 2023-06-28T20:07:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.546+0000 7f5298845700 1 --2- 172.21.15.43:0/8756410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52940d10f0 0x7f529409bd60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f528c007e60 tx=0x7f528c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.547+0000 7f5290ff9700 1 -- 172.21.15.43:0/8756410 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c00ed30 con 0x7f52940d10f0 2023-06-28T20:07:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.547+0000 7f5290ff9700 1 -- 172.21.15.43:0/8756410 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528c014420 con 0x7f52940d10f0 2023-06-28T20:07:21.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.547+0000 7f529a2a8700 1 -- 172.21.15.43:0/8756410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52940b4680 con 0x7f52940d10f0 2023-06-28T20:07:21.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.547+0000 7f5290ff9700 1 -- 172.21.15.43:0/8756410 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c0147d0 con 0x7f52940d10f0 2023-06-28T20:07:21.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.547+0000 7f529a2a8700 1 -- 172.21.15.43:0/8756410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52940b4c40 con 0x7f52940d10f0 2023-06-28T20:07:21.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.548+0000 7f529a2a8700 1 -- 172.21.15.43:0/8756410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52940b5160 con 0x7f52940d10f0 2023-06-28T20:07:21.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.548+0000 7f5290ff9700 1 -- 172.21.15.43:0/8756410 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f528c014930 con 0x7f52940d10f0 2023-06-28T20:07:21.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.548+0000 7f5290ff9700 1 --2- 172.21.15.43:0/8756410 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f528006ace0 0x7f528006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:21.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.549+0000 7f5290ff9700 1 -- 172.21.15.43:0/8756410 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f528c0899d0 con 0x7f52940d10f0 2023-06-28T20:07:21.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.549+0000 7f5293fff700 1 --2- 172.21.15.43:0/8756410 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f528006ace0 0x7f528006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:21.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.549+0000 7f5293fff700 1 --2- 172.21.15.43:0/8756410 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f528006ace0 0x7f528006d190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f529404e4a0 tx=0x7f5284008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:21.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.553+0000 7f5290ff9700 1 -- 172.21.15.43:0/8756410 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f528c054b10 con 0x7f52940d10f0 2023-06-28T20:07:21.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.751+0000 7f529a2a8700 1 -- 172.21.15.43:0/8756410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52940728b0 con 0x7f52940d10f0 2023-06-28T20:07:21.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.752+0000 7f5290ff9700 1 -- 172.21.15.43:0/8756410 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18234 (secure 0 0 0) 0x7f528c058130 con 0x7f52940d10f0 2023-06-28T20:07:21.758 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:21.758 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":20,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:07:20.788015+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:07:18.624759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-28T20:07:19.760017+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:07:21.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.754+0000 7f529a2a8700 1 -- 172.21.15.43:0/8756410 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f528006ace0 msgr2=0x7f528006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:21.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.754+0000 7f529a2a8700 1 --2- 172.21.15.43:0/8756410 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f528006ace0 0x7f528006d190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f529404e4a0 tx=0x7f5284008070 comp rx=0 tx=0).stop 2023-06-28T20:07:21.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.754+0000 7f529a2a8700 1 -- 172.21.15.43:0/8756410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52940d10f0 msgr2=0x7f529409bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:21.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.754+0000 7f529a2a8700 1 --2- 172.21.15.43:0/8756410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52940d10f0 0x7f529409bd60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f528c007e60 tx=0x7f528c00e3c0 comp rx=0 tx=0).stop 2023-06-28T20:07:21.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.755+0000 7f529a2a8700 1 -- 172.21.15.43:0/8756410 shutdown_connections 2023-06-28T20:07:21.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.755+0000 7f529a2a8700 1 --2- 172.21.15.43:0/8756410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52940d10f0 0x7f529409bd60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:21.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.755+0000 7f529a2a8700 1 --2- 172.21.15.43:0/8756410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52940cb8e0 0x7f52940b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:21.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.755+0000 7f529a2a8700 1 --2- 172.21.15.43:0/8756410 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f528006ace0 0x7f528006d190 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:21.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.755+0000 7f529a2a8700 1 --2- 172.21.15.43:0/8756410 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52940cedf0 0x7f5294159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:21.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.755+0000 7f529a2a8700 1 -- 172.21.15.43:0/8756410 >> 172.21.15.43:0/8756410 conn(0x7f52940664b0 msgr2=0x7f52940c9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:21.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.755+0000 7f529a2a8700 1 -- 172.21.15.43:0/8756410 shutdown_connections 2023-06-28T20:07:21.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.755+0000 7f529a2a8700 1 -- 172.21.15.43:0/8756410 wait complete. 2023-06-28T20:07:21.770 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:07:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.895+0000 7ff0570c8700 1 Processor -- start 2023-06-28T20:07:21.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.895+0000 7ff0570c8700 1 -- start start 2023-06-28T20:07:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.896+0000 7ff0570c8700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff0500d10f0 0x7ff0500ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.896+0000 7ff0570c8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0500cbab0 0x7ff0500b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.896+0000 7ff0570c8700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff05015ab50 con 0x7ff0500cedf0 2023-06-28T20:07:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.896+0000 7ff0570c8700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0500cbf30 con 0x7ff0500d10f0 2023-06-28T20:07:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.896+0000 7ff0570c8700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500cf260 con 0x7ff0500cbab0 2023-06-28T20:07:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.896+0000 7ff055665700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0500cbab0 0x7ff0500b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.896+0000 7ff054e64700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff0500d10f0 0x7ff0500ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.896+0000 7ff055665700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0500cbab0 0x7ff0500b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.43:36818/0 (socket says 172.21.15.43:36818) 2023-06-28T20:07:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.896+0000 7ff04ffff700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7ff0500cedf0 0x7ff0500cb3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:41152/0 (socket says 172.21.15.43:41152) 2023-06-28T20:07:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.896+0000 7ff055665700 1 -- 172.21.15.43:0/1024911287 learned_addr learned my addr 172.21.15.43:0/1024911287 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.896+0000 7ff054e64700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff0500d10f0 0x7ff0500ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:38906/0 (socket says 172.21.15.43:38906) 2023-06-28T20:07:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.896+0000 7ff055665700 1 -- 172.21.15.43:0/1024911287 >> v2:172.21.15.178:3300/0 conn(0x7ff0500d10f0 msgr2=0x7ff0500ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.896+0000 7ff055665700 1 --2- 172.21.15.43:0/1024911287 >> v2:172.21.15.178:3300/0 conn(0x7ff0500d10f0 0x7ff0500ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:21.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.896+0000 7ff055665700 1 -- 172.21.15.43:0/1024911287 >> v1:172.21.15.178:6789/0 conn(0x7ff0500cedf0 legacy=0x7ff0500cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:21.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.896+0000 7ff055665700 1 -- 172.21.15.43:0/1024911287 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0500cd1b0 con 0x7ff0500cbab0 2023-06-28T20:07:21.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.896+0000 7ff054e64700 1 --2- 172.21.15.43:0/1024911287 >> v2:172.21.15.178:3300/0 conn(0x7ff0500d10f0 0x7ff0500ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:21.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.897+0000 7ff055665700 1 --2- 172.21.15.43:0/1024911287 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0500cbab0 0x7ff0500b0600 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff048009e50 tx=0x7ff04800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ea53d326a14e92f server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:21.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.897+0000 7ff04f7fe700 1 -- 172.21.15.43:0/1024911287 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04800e070 con 0x7ff0500cbab0 2023-06-28T20:07:21.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.897+0000 7ff04f7fe700 1 -- 172.21.15.43:0/1024911287 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04800b910 con 0x7ff0500cbab0 2023-06-28T20:07:21.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.897+0000 7ff04f7fe700 1 -- 172.21.15.43:0/1024911287 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04800bcc0 con 0x7ff0500cbab0 2023-06-28T20:07:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.897+0000 7ff0570c8700 1 -- 172.21.15.43:0/1024911287 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0500cbab0 msgr2=0x7ff0500b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.897+0000 7ff0570c8700 1 --2- 172.21.15.43:0/1024911287 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0500cbab0 0x7ff0500b0600 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff048009e50 tx=0x7ff04800b330 comp rx=0 tx=0).stop 2023-06-28T20:07:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.897+0000 7ff0570c8700 1 -- 172.21.15.43:0/1024911287 shutdown_connections 2023-06-28T20:07:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.897+0000 7ff0570c8700 1 --2- 172.21.15.43:0/1024911287 >> v2:172.21.15.178:3300/0 conn(0x7ff0500d10f0 0x7ff0500ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.897+0000 7ff0570c8700 1 --2- 172.21.15.43:0/1024911287 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0500cbab0 0x7ff0500b0600 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:21.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.897+0000 7ff0570c8700 1 -- 172.21.15.43:0/1024911287 >> 172.21.15.43:0/1024911287 conn(0x7ff0500664b0 msgr2=0x7ff05009a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.897+0000 7ff0570c8700 1 -- 172.21.15.43:0/1024911287 shutdown_connections 2023-06-28T20:07:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.897+0000 7ff0570c8700 1 -- 172.21.15.43:0/1024911287 wait complete. 2023-06-28T20:07:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.899+0000 7ff0570c8700 1 Processor -- start 2023-06-28T20:07:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.899+0000 7ff0570c8700 1 -- start start 2023-06-28T20:07:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.899+0000 7ff0570c8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0500cbab0 0x7ff0500cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.900+0000 7ff0570c8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0500cedf0 0x7ff0500c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:21.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.900+0000 7ff0570c8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0500d10f0 0x7ff0500b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:21.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.900+0000 7ff0570c8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0501497b0 con 0x7ff0500d10f0 2023-06-28T20:07:21.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.900+0000 7ff0570c8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500cb0b0 con 0x7ff0500cbab0 2023-06-28T20:07:21.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.900+0000 7ff0570c8700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff050148be0 con 0x7ff0500cedf0 2023-06-28T20:07:21.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.900+0000 7ff055665700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0500d10f0 0x7ff0500b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:21.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.900+0000 7ff055665700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0500d10f0 0x7ff0500b3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52852/0 (socket says 172.21.15.43:52852) 2023-06-28T20:07:21.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.900+0000 7ff055665700 1 -- 172.21.15.43:0/3653576729 learned_addr learned my addr 172.21.15.43:0/3653576729 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:21.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.900+0000 7ff054e64700 1 --2- 172.21.15.43:0/3653576729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0500cbab0 0x7ff0500cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:21.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.900+0000 7ff04ffff700 1 --2- 172.21.15.43:0/3653576729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0500cedf0 0x7ff0500c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:21.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.900+0000 7ff055665700 1 -- 172.21.15.43:0/3653576729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0500cedf0 msgr2=0x7ff0500c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:21.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.900+0000 7ff055665700 1 --2- 172.21.15.43:0/3653576729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0500cedf0 0x7ff0500c6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:21.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.900+0000 7ff055665700 1 -- 172.21.15.43:0/3653576729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0500cbab0 msgr2=0x7ff0500cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:21.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.900+0000 7ff055665700 1 --2- 172.21.15.43:0/3653576729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0500cbab0 0x7ff0500cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:21.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.900+0000 7ff055665700 1 -- 172.21.15.43:0/3653576729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff044009430 con 0x7ff0500d10f0 2023-06-28T20:07:21.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.900+0000 7ff054e64700 1 --2- 172.21.15.43:0/3653576729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0500cbab0 0x7ff0500cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:21.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.900+0000 7ff055665700 1 --2- 172.21.15.43:0/3653576729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0500d10f0 0x7ff0500b3b10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff0480042a0 tx=0x7ff048004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:21.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.900+0000 7ff04d7fa700 1 -- 172.21.15.43:0/3653576729 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0480048d0 con 0x7ff0500d10f0 2023-06-28T20:07:21.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.900+0000 7ff04d7fa700 1 -- 172.21.15.43:0/3653576729 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0480099b0 con 0x7ff0500d10f0 2023-06-28T20:07:21.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.900+0000 7ff0570c8700 1 -- 172.21.15.43:0/3653576729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff048009b30 con 0x7ff0500d10f0 2023-06-28T20:07:21.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.900+0000 7ff04d7fa700 1 -- 172.21.15.43:0/3653576729 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04801c680 con 0x7ff0500d10f0 2023-06-28T20:07:21.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.900+0000 7ff0570c8700 1 -- 172.21.15.43:0/3653576729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff050149160 con 0x7ff0500d10f0 2023-06-28T20:07:21.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.901+0000 7ff0570c8700 1 -- 172.21.15.43:0/3653576729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0500728b0 con 0x7ff0500d10f0 2023-06-28T20:07:21.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.902+0000 7ff04d7fa700 1 -- 172.21.15.43:0/3653576729 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff04801c7e0 con 0x7ff0500d10f0 2023-06-28T20:07:21.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.902+0000 7ff04d7fa700 1 --2- 172.21.15.43:0/3653576729 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff03406ace0 0x7ff03406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:21.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.902+0000 7ff04d7fa700 1 -- 172.21.15.43:0/3653576729 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff04808ac10 con 0x7ff0500d10f0 2023-06-28T20:07:21.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.902+0000 7ff054e64700 1 --2- 172.21.15.43:0/3653576729 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff03406ace0 0x7ff03406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:21.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.903+0000 7ff054e64700 1 --2- 172.21.15.43:0/3653576729 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff03406ace0 0x7ff03406d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff0440113d0 tx=0x7ff0440091c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:21.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:21.907+0000 7ff04d7fa700 1 -- 172.21.15.43:0/3653576729 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff048055d50 con 0x7ff0500d10f0 2023-06-28T20:07:22.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.110+0000 7ff0570c8700 1 -- 172.21.15.43:0/3653576729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0501498f0 con 0x7ff0500d10f0 2023-06-28T20:07:22.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.111+0000 7ff04d7fa700 1 -- 172.21.15.43:0/3653576729 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18234 (secure 0 0 0) 0x7ff048059370 con 0x7ff0500d10f0 2023-06-28T20:07:22.114 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:22.114 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":20,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:07:20.788015+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:07:18.624759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-28T20:07:19.760017+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:07:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.113+0000 7ff0570c8700 1 -- 172.21.15.43:0/3653576729 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff03406ace0 msgr2=0x7ff03406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.113+0000 7ff0570c8700 1 --2- 172.21.15.43:0/3653576729 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff03406ace0 0x7ff03406d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff0440113d0 tx=0x7ff0440091c0 comp rx=0 tx=0).stop 2023-06-28T20:07:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.113+0000 7ff0570c8700 1 -- 172.21.15.43:0/3653576729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0500d10f0 msgr2=0x7ff0500b3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.113+0000 7ff0570c8700 1 --2- 172.21.15.43:0/3653576729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0500d10f0 0x7ff0500b3b10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff0480042a0 tx=0x7ff048004ad0 comp rx=0 tx=0).stop 2023-06-28T20:07:22.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.113+0000 7ff0570c8700 1 -- 172.21.15.43:0/3653576729 shutdown_connections 2023-06-28T20:07:22.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.113+0000 7ff0570c8700 1 --2- 172.21.15.43:0/3653576729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0500d10f0 0x7ff0500b3b10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:22.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.113+0000 7ff0570c8700 1 --2- 172.21.15.43:0/3653576729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0500cbab0 0x7ff0500cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:22.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.113+0000 7ff0570c8700 1 --2- 172.21.15.43:0/3653576729 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff03406ace0 0x7ff03406d190 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:22.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.113+0000 7ff0570c8700 1 --2- 172.21.15.43:0/3653576729 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff0500cedf0 0x7ff0500c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:22.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.113+0000 7ff0570c8700 1 -- 172.21.15.43:0/3653576729 >> 172.21.15.43:0/3653576729 conn(0x7ff0500664b0 msgr2=0x7ff0500cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:22.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.113+0000 7ff0570c8700 1 -- 172.21.15.43:0/3653576729 shutdown_connections 2023-06-28T20:07:22.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.113+0000 7ff0570c8700 1 -- 172.21.15.43:0/3653576729 wait complete. 2023-06-28T20:07:22.129 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-06-28T20:07:22.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.254+0000 7f56f90f0700 1 Processor -- start 2023-06-28T20:07:22.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.254+0000 7f56f90f0700 1 -- start start 2023-06-28T20:07:22.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.255+0000 7f56f90f0700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f40ceee0 0x7f56f406d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:22.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.255+0000 7f56f90f0700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f40ae950 con 0x7f56f40afdf0 2023-06-28T20:07:22.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.255+0000 7f56f90f0700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f415a9c0 con 0x7f56f415a5f0 2023-06-28T20:07:22.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.255+0000 7f56f90f0700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f40add40 con 0x7f56f40ceee0 2023-06-28T20:07:22.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.255+0000 7f56f359e700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f56f40afdf0 0x7f56f406d850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42046/0 (socket says 172.21.15.43:42046) 2023-06-28T20:07:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.255+0000 7f56f359e700 1 -- 172.21.15.43:0/338977398 learned_addr learned my addr 172.21.15.43:0/338977398 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.255+0000 7f56f259c700 1 --2- 172.21.15.43:0/338977398 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f40ceee0 0x7f56f406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.255+0000 7f56f1d9b700 1 -- 172.21.15.43:0/338977398 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 140214455 0 0) 0x7f56f40ae950 con 0x7f56f40afdf0 2023-06-28T20:07:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.255+0000 7f56f1d9b700 1 -- 172.21.15.43:0/338977398 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56dc0035b0 con 0x7f56f40afdf0 2023-06-28T20:07:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.255+0000 7f56f1d9b700 1 -- 172.21.15.43:0/338977398 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3800359234 0 0) 0x7f56dc0035b0 con 0x7f56f40afdf0 2023-06-28T20:07:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.255+0000 7f56f1d9b700 1 -- 172.21.15.43:0/338977398 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f40ceee0 msgr2=0x7f56f406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.255+0000 7f56f1d9b700 1 --2- 172.21.15.43:0/338977398 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f40ceee0 0x7f56f406d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.255+0000 7f56f1d9b700 1 -- 172.21.15.43:0/338977398 >> v1:172.21.15.178:6789/0 conn(0x7f56f415a5f0 legacy=0x7f56f40ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.255+0000 7f56f1d9b700 1 -- 172.21.15.43:0/338977398 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f40ae020 con 0x7f56f40afdf0 2023-06-28T20:07:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.255+0000 7f56f1d9b700 1 -- 172.21.15.43:0/338977398 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f56ec0019f0 con 0x7f56f40afdf0 2023-06-28T20:07:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.255+0000 7f56f1d9b700 1 -- 172.21.15.43:0/338977398 >> v1:172.21.15.43:6789/0 conn(0x7f56f40afdf0 legacy=0x7f56f406d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.255+0000 7f56f1d9b700 1 --2- 172.21.15.43:0/338977398 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f40ceee0 0x7f56dc003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.255+0000 7f56f1d9b700 1 --2- 172.21.15.43:0/338977398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56dc003f70 0x7f56dc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.255+0000 7f56f1d9b700 1 --2- 172.21.15.43:0/338977398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56dc004d90 0x7f56dc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.255+0000 7f56f1d9b700 1 -- 172.21.15.43:0/338977398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f40add40 con 0x7f56dc004d90 2023-06-28T20:07:22.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.255+0000 7f56f1d9b700 1 -- 172.21.15.43:0/338977398 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc00fe50 con 0x7f56dc003f70 2023-06-28T20:07:22.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.255+0000 7f56f1d9b700 1 -- 172.21.15.43:0/338977398 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc009a00 con 0x7f56f40ceee0 2023-06-28T20:07:22.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.255+0000 7f56f359e700 1 --2- 172.21.15.43:0/338977398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56dc004d90 0x7f56dc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:22.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.256+0000 7f56f259c700 1 --2- 172.21.15.43:0/338977398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56dc003f70 0x7f56dc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:22.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.256+0000 7f56f2d9d700 1 --2- 172.21.15.43:0/338977398 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f40ceee0 0x7f56dc003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:22.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.256+0000 7f56f359e700 1 -- 172.21.15.43:0/338977398 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f40ceee0 msgr2=0x7f56dc003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:22.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.256+0000 7f56f359e700 1 --2- 172.21.15.43:0/338977398 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f40ceee0 0x7f56dc003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:22.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.256+0000 7f56f359e700 1 -- 172.21.15.43:0/338977398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56dc003f70 msgr2=0x7f56dc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:22.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.256+0000 7f56f359e700 1 --2- 172.21.15.43:0/338977398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56dc003f70 0x7f56dc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:22.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.256+0000 7f56f359e700 1 -- 172.21.15.43:0/338977398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56dc0035b0 con 0x7f56dc004d90 2023-06-28T20:07:22.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.256+0000 7f56f359e700 1 --2- 172.21.15.43:0/338977398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56dc004d90 0x7f56dc005200 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00a830 tx=0x7f56dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a857b4478dd144 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:22.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.256+0000 7f56f1d9b700 1 -- 172.21.15.43:0/338977398 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec012600 con 0x7f56dc004d90 2023-06-28T20:07:22.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.256+0000 7f56f1d9b700 1 -- 172.21.15.43:0/338977398 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ec012780 con 0x7f56dc004d90 2023-06-28T20:07:22.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.256+0000 7f56f1d9b700 1 -- 172.21.15.43:0/338977398 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec012b30 con 0x7f56dc004d90 2023-06-28T20:07:22.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.257+0000 7f56f90f0700 1 -- 172.21.15.43:0/338977398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56dc004d90 msgr2=0x7f56dc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:22.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.257+0000 7f56f90f0700 1 --2- 172.21.15.43:0/338977398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56dc004d90 0x7f56dc005200 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00a830 tx=0x7f56dc005880 comp rx=0 tx=0).stop 2023-06-28T20:07:22.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.257+0000 7f56f359e700 1 -- 172.21.15.43:0/338977398 reap_dead start 2023-06-28T20:07:22.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.257+0000 7f56f90f0700 1 -- 172.21.15.43:0/338977398 shutdown_connections 2023-06-28T20:07:22.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.257+0000 7f56f90f0700 1 -- 172.21.15.43:0/338977398 >> 172.21.15.43:0/338977398 conn(0x7f56f40664b0 msgr2=0x7f56f40d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:22.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.257+0000 7f56f90f0700 1 -- 172.21.15.43:0/338977398 shutdown_connections 2023-06-28T20:07:22.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.257+0000 7f56f90f0700 1 -- 172.21.15.43:0/338977398 wait complete. 2023-06-28T20:07:22.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.258+0000 7f56f90f0700 1 Processor -- start 2023-06-28T20:07:22.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.258+0000 7f56f90f0700 1 -- start start 2023-06-28T20:07:22.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.259+0000 7f56f90f0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56dc004d90 0x7f56f40c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:22.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.259+0000 7f56f90f0700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f40b6120 0x7f56f406e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:22.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.259+0000 7f56f90f0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56f40b6590 0x7f56f406e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:22.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.259+0000 7f56f90f0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f4153850 con 0x7f56dc004d90 2023-06-28T20:07:22.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.259+0000 7f56f90f0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f4153af0 con 0x7f56f40b6590 2023-06-28T20:07:22.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.259+0000 7f56f90f0700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f4153dd0 con 0x7f56f40b6120 2023-06-28T20:07:22.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.259+0000 7f56f2d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56dc004d90 0x7f56f40c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:22.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.259+0000 7f56f2d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56dc004d90 0x7f56f40c6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52868/0 (socket says 172.21.15.43:52868) 2023-06-28T20:07:22.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.259+0000 7f56f2d9d700 1 -- 172.21.15.43:0/4191139240 learned_addr learned my addr 172.21.15.43:0/4191139240 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:22.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.259+0000 7f56f259c700 1 --2- 172.21.15.43:0/4191139240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f40b6120 0x7f56f406e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:22.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.259+0000 7f56f359e700 1 --2- 172.21.15.43:0/4191139240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56f40b6590 0x7f56f406e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:22.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.259+0000 7f56f2d9d700 1 -- 172.21.15.43:0/4191139240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f40b6120 msgr2=0x7f56f406e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:22.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.259+0000 7f56f2d9d700 1 --2- 172.21.15.43:0/4191139240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f40b6120 0x7f56f406e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:22.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.259+0000 7f56f2d9d700 1 -- 172.21.15.43:0/4191139240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56f40b6590 msgr2=0x7f56f406e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:22.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.259+0000 7f56f2d9d700 1 --2- 172.21.15.43:0/4191139240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56f40b6590 0x7f56f406e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:22.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.259+0000 7f56f2d9d700 1 -- 172.21.15.43:0/4191139240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f41540b0 con 0x7f56dc004d90 2023-06-28T20:07:22.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.260+0000 7f56f2d9d700 1 --2- 172.21.15.43:0/4191139240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56dc004d90 0x7f56f40c6920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f56e400f820 tx=0x7f56e400fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:22.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.260+0000 7f56db7fe700 1 -- 172.21.15.43:0/4191139240 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e40052e0 con 0x7f56dc004d90 2023-06-28T20:07:22.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.260+0000 7f56db7fe700 1 -- 172.21.15.43:0/4191139240 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e4005440 con 0x7f56dc004d90 2023-06-28T20:07:22.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.260+0000 7f56f90f0700 1 -- 172.21.15.43:0/4191139240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f4154390 con 0x7f56dc004d90 2023-06-28T20:07:22.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.260+0000 7f56db7fe700 1 -- 172.21.15.43:0/4191139240 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e40055b0 con 0x7f56dc004d90 2023-06-28T20:07:22.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.260+0000 7f56f90f0700 1 -- 172.21.15.43:0/4191139240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f40c6e60 con 0x7f56dc004d90 2023-06-28T20:07:22.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.261+0000 7f56f90f0700 1 -- 172.21.15.43:0/4191139240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f40728b0 con 0x7f56dc004d90 2023-06-28T20:07:22.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.261+0000 7f56db7fe700 1 -- 172.21.15.43:0/4191139240 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f56e4003910 con 0x7f56dc004d90 2023-06-28T20:07:22.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.262+0000 7f56db7fe700 1 --2- 172.21.15.43:0/4191139240 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f56e006ace0 0x7f56e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:22.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.262+0000 7f56db7fe700 1 -- 172.21.15.43:0/4191139240 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f56e4089210 con 0x7f56dc004d90 2023-06-28T20:07:22.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.262+0000 7f56f259c700 1 --2- 172.21.15.43:0/4191139240 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f56e006ace0 0x7f56e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:22.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.262+0000 7f56f259c700 1 --2- 172.21.15.43:0/4191139240 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f56e006ace0 0x7f56e006d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f56f4155120 tx=0x7f56e8009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:22.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.266+0000 7f56db7fe700 1 -- 172.21.15.43:0/4191139240 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f56e4054400 con 0x7f56dc004d90 2023-06-28T20:07:22.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:22.510+0000 7f56f90f0700 1 -- 172.21.15.43:0/4191139240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f56f4070f10 con 0x7f56dc004d90 2023-06-28T20:07:23.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.389+0000 7f56db7fe700 1 -- 172.21.15.43:0/4191139240 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7f56e4057a20 con 0x7f56dc004d90 2023-06-28T20:07:23.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.392+0000 7f56f90f0700 1 -- 172.21.15.43:0/4191139240 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f56e006ace0 msgr2=0x7f56e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:23.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.392+0000 7f56f90f0700 1 --2- 172.21.15.43:0/4191139240 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f56e006ace0 0x7f56e006d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f56f4155120 tx=0x7f56e8009210 comp rx=0 tx=0).stop 2023-06-28T20:07:23.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.392+0000 7f56f90f0700 1 -- 172.21.15.43:0/4191139240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56dc004d90 msgr2=0x7f56f40c6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:23.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.392+0000 7f56f90f0700 1 --2- 172.21.15.43:0/4191139240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56dc004d90 0x7f56f40c6920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f56e400f820 tx=0x7f56e400fb30 comp rx=0 tx=0).stop 2023-06-28T20:07:23.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.392+0000 7f56f90f0700 1 -- 172.21.15.43:0/4191139240 shutdown_connections 2023-06-28T20:07:23.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.392+0000 7f56f90f0700 1 --2- 172.21.15.43:0/4191139240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56f40b6590 0x7f56f406e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:23.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.392+0000 7f56f90f0700 1 --2- 172.21.15.43:0/4191139240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56dc004d90 0x7f56f40c6920 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:23.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.392+0000 7f56f90f0700 1 --2- 172.21.15.43:0/4191139240 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f56e006ace0 0x7f56e006d190 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:23.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.392+0000 7f56f90f0700 1 --2- 172.21.15.43:0/4191139240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f56f40b6120 0x7f56f406e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:23.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.392+0000 7f56f90f0700 1 -- 172.21.15.43:0/4191139240 >> 172.21.15.43:0/4191139240 conn(0x7f56f40664b0 msgr2=0x7f56f41598c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:23.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.392+0000 7f56f90f0700 1 -- 172.21.15.43:0/4191139240 shutdown_connections 2023-06-28T20:07:23.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.392+0000 7f56f90f0700 1 -- 172.21.15.43:0/4191139240 wait complete. 2023-06-28T20:07:23.410 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-06-28T20:07:23.410 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-06-28T20:07:23.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.542+0000 7f13a6efa700 1 Processor -- start 2023-06-28T20:07:23.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.542+0000 7f13a6efa700 1 -- start start 2023-06-28T20:07:23.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.543+0000 7f13a6efa700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a01592d0 0x7f13a00b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:23.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.543+0000 7f13a6efa700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f13a01596a0 0x7f13a00b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:23.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.543+0000 7f13a6efa700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f13a0071900 0x7f13a015d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:23.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.543+0000 7f13a6efa700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a0070b40 con 0x7f13a0071900 2023-06-28T20:07:23.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.543+0000 7f13a6efa700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a0070e20 con 0x7f13a01596a0 2023-06-28T20:07:23.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.543+0000 7f13a6efa700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a0071100 con 0x7f13a01592d0 2023-06-28T20:07:23.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.543+0000 7f13a5497700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f13a0071900 0x7f13a015d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:23.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.543+0000 7f13a5497700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f13a0071900 0x7f13a015d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52878/0 (socket says 172.21.15.43:52878) 2023-06-28T20:07:23.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.543+0000 7f13a5497700 1 -- 172.21.15.43:0/3785218050 learned_addr learned my addr 172.21.15.43:0/3785218050 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:23.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.543+0000 7f13a4c96700 1 --2- 172.21.15.43:0/3785218050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a01592d0 0x7f13a00b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:23.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.543+0000 7f139ffff700 1 --2- 172.21.15.43:0/3785218050 >> v2:172.21.15.178:3300/0 conn(0x7f13a01596a0 0x7f13a00b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:23.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.543+0000 7f13a5497700 1 -- 172.21.15.43:0/3785218050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a01592d0 msgr2=0x7f13a00b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:23.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.543+0000 7f13a5497700 1 --2- 172.21.15.43:0/3785218050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a01592d0 0x7f13a00b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:23.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.543+0000 7f13a5497700 1 -- 172.21.15.43:0/3785218050 >> v2:172.21.15.178:3300/0 conn(0x7f13a01596a0 msgr2=0x7f13a00b1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:23.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.543+0000 7f13a5497700 1 --2- 172.21.15.43:0/3785218050 >> v2:172.21.15.178:3300/0 conn(0x7f13a01596a0 0x7f13a00b1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:23.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.543+0000 7f13a5497700 1 -- 172.21.15.43:0/3785218050 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a00713e0 con 0x7f13a0071900 2023-06-28T20:07:23.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.543+0000 7f13a5497700 1 --2- 172.21.15.43:0/3785218050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a0071900 0x7f13a015d070 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f13a00d0b00 tx=0x7f139800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61772d9928ca2572 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:23.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.544+0000 7f139f7fe700 1 -- 172.21.15.43:0/3785218050 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139800e070 con 0x7f13a0071900 2023-06-28T20:07:23.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.544+0000 7f139f7fe700 1 -- 172.21.15.43:0/3785218050 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139800b9c0 con 0x7f13a0071900 2023-06-28T20:07:23.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.544+0000 7f139f7fe700 1 -- 172.21.15.43:0/3785218050 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139800bd70 con 0x7f13a0071900 2023-06-28T20:07:23.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.544+0000 7f13a6efa700 1 -- 172.21.15.43:0/3785218050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a0071900 msgr2=0x7f13a015d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:23.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.544+0000 7f13a6efa700 1 --2- 172.21.15.43:0/3785218050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a0071900 0x7f13a015d070 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f13a00d0b00 tx=0x7f139800b330 comp rx=0 tx=0).stop 2023-06-28T20:07:23.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.544+0000 7f13a6efa700 1 -- 172.21.15.43:0/3785218050 shutdown_connections 2023-06-28T20:07:23.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.544+0000 7f13a6efa700 1 --2- 172.21.15.43:0/3785218050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a0071900 0x7f13a015d070 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:23.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.544+0000 7f13a6efa700 1 --2- 172.21.15.43:0/3785218050 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a01592d0 0x7f13a00b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:23.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.544+0000 7f13a6efa700 1 --2- 172.21.15.43:0/3785218050 >> v2:172.21.15.178:3300/0 conn(0x7f13a01596a0 0x7f13a00b1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:23.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.544+0000 7f13a6efa700 1 -- 172.21.15.43:0/3785218050 >> 172.21.15.43:0/3785218050 conn(0x7f13a006b000 msgr2=0x7f13a006b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:23.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.544+0000 7f13a6efa700 1 -- 172.21.15.43:0/3785218050 shutdown_connections 2023-06-28T20:07:23.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.544+0000 7f13a6efa700 1 -- 172.21.15.43:0/3785218050 wait complete. 2023-06-28T20:07:23.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.545+0000 7f13a6efa700 1 Processor -- start 2023-06-28T20:07:23.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.545+0000 7f13a6efa700 1 -- start start 2023-06-28T20:07:23.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.546+0000 7f13a6efa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13a0071900 0x7f13a00af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:23.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.546+0000 7f13a6efa700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a01592d0 0x7f13a0157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:23.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.546+0000 7f13a6efa700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a01596a0 0x7f13a0150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:23.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.546+0000 7f13a6efa700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a01582c0 con 0x7f13a01596a0 2023-06-28T20:07:23.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.546+0000 7f13a6efa700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a00b2a70 con 0x7f13a0071900 2023-06-28T20:07:23.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.546+0000 7f13a6efa700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a0157fe0 con 0x7f13a01592d0 2023-06-28T20:07:23.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.546+0000 7f13a5497700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a01596a0 0x7f13a0150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:23.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.547+0000 7f13a5497700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a01596a0 0x7f13a0150c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52884/0 (socket says 172.21.15.43:52884) 2023-06-28T20:07:23.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.547+0000 7f13a5497700 1 -- 172.21.15.43:0/2012292785 learned_addr learned my addr 172.21.15.43:0/2012292785 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:23.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.547+0000 7f13a4c96700 1 --2- 172.21.15.43:0/2012292785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13a0071900 0x7f13a00af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:23.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.547+0000 7f139ffff700 1 --2- 172.21.15.43:0/2012292785 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a01592d0 0x7f13a0157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:23.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.547+0000 7f13a5497700 1 -- 172.21.15.43:0/2012292785 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a01592d0 msgr2=0x7f13a0157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:23.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.547+0000 7f13a5497700 1 --2- 172.21.15.43:0/2012292785 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a01592d0 0x7f13a0157600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:23.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.547+0000 7f13a5497700 1 -- 172.21.15.43:0/2012292785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13a0071900 msgr2=0x7f13a00af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:23.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.547+0000 7f13a5497700 1 --2- 172.21.15.43:0/2012292785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13a0071900 0x7f13a00af800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:23.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.547+0000 7f13a5497700 1 -- 172.21.15.43:0/2012292785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1398009b30 con 0x7f13a01596a0 2023-06-28T20:07:23.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.547+0000 7f13a5497700 1 --2- 172.21.15.43:0/2012292785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a01596a0 0x7f13a0150c80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1398012000 tx=0x7f1398004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:23.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.547+0000 7f139d7fa700 1 -- 172.21.15.43:0/2012292785 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13980099b0 con 0x7f13a01596a0 2023-06-28T20:07:23.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.547+0000 7f139d7fa700 1 -- 172.21.15.43:0/2012292785 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1398004250 con 0x7f13a01596a0 2023-06-28T20:07:23.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.547+0000 7f13a6efa700 1 -- 172.21.15.43:0/2012292785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a00b2d10 con 0x7f13a01596a0 2023-06-28T20:07:23.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.547+0000 7f139d7fa700 1 -- 172.21.15.43:0/2012292785 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398004600 con 0x7f13a01596a0 2023-06-28T20:07:23.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.547+0000 7f13a6efa700 1 -- 172.21.15.43:0/2012292785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a00afd40 con 0x7f13a01596a0 2023-06-28T20:07:23.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.548+0000 7f13a6efa700 1 -- 172.21.15.43:0/2012292785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a00cc3e0 con 0x7f13a01596a0 2023-06-28T20:07:23.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.549+0000 7f139d7fa700 1 -- 172.21.15.43:0/2012292785 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1398004760 con 0x7f13a01596a0 2023-06-28T20:07:23.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.549+0000 7f139d7fa700 1 --2- 172.21.15.43:0/2012292785 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f138406ace0 0x7f138406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:23.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.549+0000 7f13a4c96700 1 --2- 172.21.15.43:0/2012292785 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f138406ace0 0x7f138406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:23.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.549+0000 7f139d7fa700 1 -- 172.21.15.43:0/2012292785 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f139808a840 con 0x7f13a01596a0 2023-06-28T20:07:23.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.549+0000 7f13a4c96700 1 --2- 172.21.15.43:0/2012292785 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f138406ace0 0x7f138406d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1394009950 tx=0x7f1394008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:23.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.554+0000 7f139d7fa700 1 -- 172.21.15.43:0/2012292785 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1398055980 con 0x7f13a01596a0 2023-06-28T20:07:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:23.803+0000 7f13a6efa700 1 -- 172.21.15.43:0/2012292785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f13a00cf340 con 0x7f13a01596a0 2023-06-28T20:07:24.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.394+0000 7f139d7fa700 1 -- 172.21.15.43:0/2012292785 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7f13980200c0 con 0x7f13a01596a0 2023-06-28T20:07:24.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.397+0000 7f13a6efa700 1 -- 172.21.15.43:0/2012292785 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f138406ace0 msgr2=0x7f138406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:24.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.397+0000 7f13a6efa700 1 --2- 172.21.15.43:0/2012292785 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f138406ace0 0x7f138406d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1394009950 tx=0x7f1394008040 comp rx=0 tx=0).stop 2023-06-28T20:07:24.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.397+0000 7f13a6efa700 1 -- 172.21.15.43:0/2012292785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a01596a0 msgr2=0x7f13a0150c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:24.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.397+0000 7f13a6efa700 1 --2- 172.21.15.43:0/2012292785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a01596a0 0x7f13a0150c80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1398012000 tx=0x7f1398004ad0 comp rx=0 tx=0).stop 2023-06-28T20:07:24.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.397+0000 7f13a6efa700 1 -- 172.21.15.43:0/2012292785 shutdown_connections 2023-06-28T20:07:24.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.397+0000 7f13a6efa700 1 --2- 172.21.15.43:0/2012292785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a01596a0 0x7f13a0150c80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:24.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.397+0000 7f13a6efa700 1 --2- 172.21.15.43:0/2012292785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13a0071900 0x7f13a00af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:24.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.397+0000 7f13a6efa700 1 --2- 172.21.15.43:0/2012292785 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f138406ace0 0x7f138406d190 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:24.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.397+0000 7f13a6efa700 1 --2- 172.21.15.43:0/2012292785 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13a01592d0 0x7f13a0157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:24.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.397+0000 7f13a6efa700 1 -- 172.21.15.43:0/2012292785 >> 172.21.15.43:0/2012292785 conn(0x7f13a006b000 msgr2=0x7f13a00d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:24.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.397+0000 7f13a6efa700 1 -- 172.21.15.43:0/2012292785 shutdown_connections 2023-06-28T20:07:24.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.397+0000 7f13a6efa700 1 -- 172.21.15.43:0/2012292785 wait complete. 2023-06-28T20:07:24.415 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-06-28T20:07:24.415 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-28T20:07:24.434 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-28T20:07:24.460 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-28T20:07:24.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.545+0000 7f105c5eb700 1 Processor -- start 2023-06-28T20:07:24.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.545+0000 7f105c5eb700 1 -- start start 2023-06-28T20:07:24.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.546+0000 7f105c5eb700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f105406c8b0 0x7f105415e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:24.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.546+0000 7f105c5eb700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f105415d680 0x7f105415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:24.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.546+0000 7f105c5eb700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f105415def0 con 0x7f105415c4e0 2023-06-28T20:07:24.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.546+0000 7f105c5eb700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10540cf710 con 0x7f105415d680 2023-06-28T20:07:24.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.546+0000 7f105c5eb700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10540cf150 con 0x7f105406c8b0 2023-06-28T20:07:24.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.546+0000 7f105ab88700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f105415d680 0x7f105415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:24.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.546+0000 7f105ab88700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f105415d680 0x7f105415e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52892/0 (socket says 172.21.15.43:52892) 2023-06-28T20:07:24.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.546+0000 7f105ab88700 1 -- 172.21.15.43:0/4263235000 learned_addr learned my addr 172.21.15.43:0/4263235000 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:24.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.546+0000 7f1059b86700 1 --2- 172.21.15.43:0/4263235000 >> v2:172.21.15.178:3300/0 conn(0x7f105406c8b0 0x7f105415e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:24.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.546+0000 7f105ab88700 1 -- 172.21.15.43:0/4263235000 >> v2:172.21.15.178:3300/0 conn(0x7f105406c8b0 msgr2=0x7f105415e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:24.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.546+0000 7f105ab88700 1 --2- 172.21.15.43:0/4263235000 >> v2:172.21.15.178:3300/0 conn(0x7f105406c8b0 0x7f105415e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:24.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.546+0000 7f105ab88700 1 -- 172.21.15.43:0/4263235000 >> v1:172.21.15.178:6789/0 conn(0x7f105415c4e0 legacy=0x7f105406a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:24.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.546+0000 7f105ab88700 1 -- 172.21.15.43:0/4263235000 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10540cf430 con 0x7f105415d680 2023-06-28T20:07:24.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.546+0000 7f105ab88700 1 --2- 172.21.15.43:0/4263235000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f105415d680 0x7f105415e8c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f10540d3b50 tx=0x7f104c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680d3a4758a6e07f server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:24.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.546+0000 7f1059385700 1 -- 172.21.15.43:0/4263235000 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c00e070 con 0x7f105415d680 2023-06-28T20:07:24.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.546+0000 7f1059385700 1 -- 172.21.15.43:0/4263235000 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104c00b910 con 0x7f105415d680 2023-06-28T20:07:24.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.546+0000 7f1059385700 1 -- 172.21.15.43:0/4263235000 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c00bcc0 con 0x7f105415d680 2023-06-28T20:07:24.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.547+0000 7f105c5eb700 1 -- 172.21.15.43:0/4263235000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f105415d680 msgr2=0x7f105415e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:24.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.547+0000 7f105c5eb700 1 --2- 172.21.15.43:0/4263235000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f105415d680 0x7f105415e8c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f10540d3b50 tx=0x7f104c00b330 comp rx=0 tx=0).stop 2023-06-28T20:07:24.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.547+0000 7f105c5eb700 1 -- 172.21.15.43:0/4263235000 shutdown_connections 2023-06-28T20:07:24.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.547+0000 7f105c5eb700 1 --2- 172.21.15.43:0/4263235000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f105415d680 0x7f105415e8c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:24.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.547+0000 7f105c5eb700 1 --2- 172.21.15.43:0/4263235000 >> v2:172.21.15.178:3300/0 conn(0x7f105406c8b0 0x7f105415e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:24.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.547+0000 7f105c5eb700 1 -- 172.21.15.43:0/4263235000 >> 172.21.15.43:0/4263235000 conn(0x7f10540d2eb0 msgr2=0x7f10540d32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:24.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.547+0000 7f105c5eb700 1 -- 172.21.15.43:0/4263235000 shutdown_connections 2023-06-28T20:07:24.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.547+0000 7f105c5eb700 1 -- 172.21.15.43:0/4263235000 wait complete. 2023-06-28T20:07:24.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.548+0000 7f105c5eb700 1 Processor -- start 2023-06-28T20:07:24.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.548+0000 7f105c5eb700 1 -- start start 2023-06-28T20:07:24.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.549+0000 7f105c5eb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f105406c8b0 0x7f1054151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:24.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.549+0000 7f105c5eb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f105415c4e0 0x7f10540b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:24.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.549+0000 7f105c5eb700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f105415d680 0x7f10540b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:24.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.549+0000 7f105c5eb700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10540b40e0 con 0x7f105406c8b0 2023-06-28T20:07:24.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.549+0000 7f105c5eb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1054152950 con 0x7f105415c4e0 2023-06-28T20:07:24.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.549+0000 7f105c5eb700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10540b3460 con 0x7f105415d680 2023-06-28T20:07:24.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.549+0000 7f105a387700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f105406c8b0 0x7f1054151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:24.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.549+0000 7f105a387700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f105406c8b0 0x7f1054151dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52894/0 (socket says 172.21.15.43:52894) 2023-06-28T20:07:24.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.549+0000 7f105a387700 1 -- 172.21.15.43:0/1134794007 learned_addr learned my addr 172.21.15.43:0/1134794007 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:24.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.549+0000 7f1059b86700 1 --2- 172.21.15.43:0/1134794007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f105415c4e0 0x7f10540b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:24.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.549+0000 7f105ab88700 1 --2- 172.21.15.43:0/1134794007 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f105415d680 0x7f10540b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:24.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.549+0000 7f105a387700 1 -- 172.21.15.43:0/1134794007 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f105415d680 msgr2=0x7f10540b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:24.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.549+0000 7f105a387700 1 --2- 172.21.15.43:0/1134794007 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f105415d680 0x7f10540b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:24.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.549+0000 7f105a387700 1 -- 172.21.15.43:0/1134794007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f105415c4e0 msgr2=0x7f10540b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:24.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.549+0000 7f105a387700 1 --2- 172.21.15.43:0/1134794007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f105415c4e0 0x7f10540b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:24.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.549+0000 7f105a387700 1 -- 172.21.15.43:0/1134794007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104c009b30 con 0x7f105406c8b0 2023-06-28T20:07:24.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.549+0000 7f105a387700 1 --2- 172.21.15.43:0/1134794007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f105406c8b0 0x7f1054151dd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f105000c4e0 tx=0x7f105000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:24.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.550+0000 7f104affd700 1 -- 172.21.15.43:0/1134794007 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10500052d0 con 0x7f105406c8b0 2023-06-28T20:07:24.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.550+0000 7f104affd700 1 -- 172.21.15.43:0/1134794007 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1050005430 con 0x7f105406c8b0 2023-06-28T20:07:24.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.550+0000 7f105c5eb700 1 -- 172.21.15.43:0/1134794007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10540b3740 con 0x7f105406c8b0 2023-06-28T20:07:24.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.550+0000 7f104affd700 1 -- 172.21.15.43:0/1134794007 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10500038c0 con 0x7f105406c8b0 2023-06-28T20:07:24.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.550+0000 7f105c5eb700 1 -- 172.21.15.43:0/1134794007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10540b3d00 con 0x7f105406c8b0 2023-06-28T20:07:24.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.551+0000 7f105c5eb700 1 -- 172.21.15.43:0/1134794007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10540728b0 con 0x7f105406c8b0 2023-06-28T20:07:24.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.551+0000 7f104affd700 1 -- 172.21.15.43:0/1134794007 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1050003a20 con 0x7f105406c8b0 2023-06-28T20:07:24.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.552+0000 7f104affd700 1 --2- 172.21.15.43:0/1134794007 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f103806ace0 0x7f103806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:24.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.552+0000 7f104affd700 1 -- 172.21.15.43:0/1134794007 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1050088ce0 con 0x7f105406c8b0 2023-06-28T20:07:24.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.552+0000 7f1059b86700 1 --2- 172.21.15.43:0/1134794007 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f103806ace0 0x7f103806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:24.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.552+0000 7f1059b86700 1 --2- 172.21.15.43:0/1134794007 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f103806ace0 0x7f103806d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1054152b70 tx=0x7f1044008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:24.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.556+0000 7f104affd700 1 -- 172.21.15.43:0/1134794007 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1050053f00 con 0x7f105406c8b0 2023-06-28T20:07:24.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.798+0000 7f105c5eb700 1 -- 172.21.15.43:0/1134794007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10540d0760 con 0x7f105406c8b0 2023-06-28T20:07:24.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.799+0000 7f104affd700 1 -- 172.21.15.43:0/1134794007 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4665 (secure 0 0 0) 0x7f1050057520 con 0x7f105406c8b0 2023-06-28T20:07:24.802 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:24.802 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4033115951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4033115951},{"type":"v1","addr":"172.21.15.178:6835","nonce":4033115951}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/65569869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":65569869},{"type":"v1","addr":"172.21.15.43:6837","nonce":65569869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/1016213057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016213057},{"type":"v1","addr":"172.21.15.178:6834","nonce":1016213057}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-28T20:07:20.788417+0000","modified":"2023-06-28T20:07:24.388714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/3514544484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3514544484},{"type":"v1","addr":"172.21.15.43:6835","nonce":3514544484}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-28T20:07:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.801+0000 7f105c5eb700 1 -- 172.21.15.43:0/1134794007 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f103806ace0 msgr2=0x7f103806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:24.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.801+0000 7f105c5eb700 1 --2- 172.21.15.43:0/1134794007 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f103806ace0 0x7f103806d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1054152b70 tx=0x7f1044008040 comp rx=0 tx=0).stop 2023-06-28T20:07:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.801+0000 7f105c5eb700 1 -- 172.21.15.43:0/1134794007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f105406c8b0 msgr2=0x7f1054151dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.801+0000 7f105c5eb700 1 --2- 172.21.15.43:0/1134794007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f105406c8b0 0x7f1054151dd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f105000c4e0 tx=0x7f105000c8a0 comp rx=0 tx=0).stop 2023-06-28T20:07:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.801+0000 7f105c5eb700 1 -- 172.21.15.43:0/1134794007 shutdown_connections 2023-06-28T20:07:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.801+0000 7f105c5eb700 1 --2- 172.21.15.43:0/1134794007 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f103806ace0 0x7f103806d190 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.801+0000 7f105c5eb700 1 --2- 172.21.15.43:0/1134794007 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f105415d680 0x7f10540b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.801+0000 7f105c5eb700 1 --2- 172.21.15.43:0/1134794007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f105406c8b0 0x7f1054151dd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:24.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.801+0000 7f105c5eb700 1 --2- 172.21.15.43:0/1134794007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f105415c4e0 0x7f10540b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.801+0000 7f105c5eb700 1 -- 172.21.15.43:0/1134794007 >> 172.21.15.43:0/1134794007 conn(0x7f10540d2eb0 msgr2=0x7f10540b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.801+0000 7f105c5eb700 1 -- 172.21.15.43:0/1134794007 shutdown_connections 2023-06-28T20:07:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.801+0000 7f105c5eb700 1 -- 172.21.15.43:0/1134794007 wait complete. 2023-06-28T20:07:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 7 2023-06-28T20:07:24.816 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:07:24.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.976+0000 7f8663c7c700 1 Processor -- start 2023-06-28T20:07:24.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.977+0000 7f8663c7c700 1 -- start start 2023-06-28T20:07:24.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.977+0000 7f8663c7c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f865c0cefc0 0x7f865c0cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:24.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.977+0000 7f8663c7c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f865c0cb8e0 0x7f865c0b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:24.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.977+0000 7f8663c7c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f865c15ab50 con 0x7f865c0d10f0 2023-06-28T20:07:24.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.977+0000 7f8663c7c700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f865c0cd1b0 con 0x7f865c0cefc0 2023-06-28T20:07:24.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.977+0000 7f8663c7c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c0cc110 con 0x7f865c0cb8e0 2023-06-28T20:07:24.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.977+0000 7f8661217700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f865c0cefc0 0x7f865c0cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:24.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.978+0000 7f8661a18700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f865c0d10f0 0x7f865c0ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42088/0 (socket says 172.21.15.43:42088) 2023-06-28T20:07:24.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.978+0000 7f8661a18700 1 -- 172.21.15.43:0/86382696 learned_addr learned my addr 172.21.15.43:0/86382696 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:24.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.978+0000 7f8661217700 1 -- 172.21.15.43:0/86382696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f865c0cb8e0 msgr2=0x7f865c0b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:24.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.978+0000 7f8662219700 1 --2- 172.21.15.43:0/86382696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f865c0cb8e0 0x7f865c0b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:24.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.978+0000 7f8661217700 1 --2- 172.21.15.43:0/86382696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f865c0cb8e0 0x7f865c0b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:24.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.978+0000 7f8661217700 1 -- 172.21.15.43:0/86382696 >> v1:172.21.15.43:6789/0 conn(0x7f865c0d10f0 legacy=0x7f865c0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:24.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.978+0000 7f8660a16700 1 -- 172.21.15.43:0/86382696 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3869903662 0 0) 0x7f865c15ab50 con 0x7f865c0d10f0 2023-06-28T20:07:24.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.978+0000 7f8661217700 1 -- 172.21.15.43:0/86382696 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c0cc3f0 con 0x7f865c0cefc0 2023-06-28T20:07:24.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.978+0000 7f8661217700 1 --2- 172.21.15.43:0/86382696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f865c0cefc0 0x7f865c0cb3a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8650009840 tx=0x7f8650009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96392820ac3aaee server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:24.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.978+0000 7f8660a16700 1 -- 172.21.15.43:0/86382696 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8650011830 con 0x7f865c0cefc0 2023-06-28T20:07:24.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.978+0000 7f8660a16700 1 -- 172.21.15.43:0/86382696 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8650011990 con 0x7f865c0cefc0 2023-06-28T20:07:24.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.978+0000 7f8660a16700 1 -- 172.21.15.43:0/86382696 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8650011d20 con 0x7f865c0cefc0 2023-06-28T20:07:24.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.979+0000 7f8663c7c700 1 -- 172.21.15.43:0/86382696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f865c0cefc0 msgr2=0x7f865c0cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:24.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.979+0000 7f8663c7c700 1 --2- 172.21.15.43:0/86382696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f865c0cefc0 0x7f865c0cb3a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8650009840 tx=0x7f8650009b50 comp rx=0 tx=0).stop 2023-06-28T20:07:24.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.979+0000 7f8663c7c700 1 -- 172.21.15.43:0/86382696 shutdown_connections 2023-06-28T20:07:24.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.979+0000 7f8663c7c700 1 --2- 172.21.15.43:0/86382696 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f865c0cb8e0 0x7f865c0b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:24.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.979+0000 7f8663c7c700 1 --2- 172.21.15.43:0/86382696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f865c0cefc0 0x7f865c0cb3a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:24.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.979+0000 7f8663c7c700 1 -- 172.21.15.43:0/86382696 >> 172.21.15.43:0/86382696 conn(0x7f865c0664b0 msgr2=0x7f865c09a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:24.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.979+0000 7f8663c7c700 1 -- 172.21.15.43:0/86382696 shutdown_connections 2023-06-28T20:07:24.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.979+0000 7f8663c7c700 1 -- 172.21.15.43:0/86382696 wait complete. 2023-06-28T20:07:24.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.980+0000 7f8663c7c700 1 Processor -- start 2023-06-28T20:07:24.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.980+0000 7f8663c7c700 1 -- start start 2023-06-28T20:07:24.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.981+0000 7f8663c7c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f865c0cb8e0 0x7f865c158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:24.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.981+0000 7f8663c7c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f865c0cefc0 0x7f865c0cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:24.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.981+0000 7f8663c7c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f865c0d10f0 0x7f865c1e3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:24.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.981+0000 7f8663c7c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c158a50 con 0x7f865c0cb8e0 2023-06-28T20:07:24.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.981+0000 7f8663c7c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c09ea80 con 0x7f865c0cefc0 2023-06-28T20:07:24.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.981+0000 7f8663c7c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c06b000 con 0x7f865c0d10f0 2023-06-28T20:07:24.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.981+0000 7f8661a18700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f865c0cb8e0 0x7f865c158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:24.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.981+0000 7f8661a18700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f865c0cb8e0 0x7f865c158510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52908/0 (socket says 172.21.15.43:52908) 2023-06-28T20:07:24.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.981+0000 7f8661a18700 1 -- 172.21.15.43:0/2764031329 learned_addr learned my addr 172.21.15.43:0/2764031329 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:24.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.981+0000 7f8661217700 1 --2- 172.21.15.43:0/2764031329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f865c0cefc0 0x7f865c0cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:24.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.981+0000 7f8662219700 1 --2- 172.21.15.43:0/2764031329 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f865c0d10f0 0x7f865c1e3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:24.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.981+0000 7f8661a18700 1 -- 172.21.15.43:0/2764031329 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f865c0d10f0 msgr2=0x7f865c1e3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:24.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.981+0000 7f8661a18700 1 --2- 172.21.15.43:0/2764031329 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f865c0d10f0 0x7f865c1e3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:24.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.981+0000 7f8661a18700 1 -- 172.21.15.43:0/2764031329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f865c0cefc0 msgr2=0x7f865c0cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:24.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.981+0000 7f8661a18700 1 --2- 172.21.15.43:0/2764031329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f865c0cefc0 0x7f865c0cada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:24.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.981+0000 7f8661a18700 1 -- 172.21.15.43:0/2764031329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86500094a0 con 0x7f865c0cb8e0 2023-06-28T20:07:24.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.981+0000 7f8661217700 1 --2- 172.21.15.43:0/2764031329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f865c0cefc0 0x7f865c0cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:24.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.981+0000 7f8661a18700 1 --2- 172.21.15.43:0/2764031329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f865c0cb8e0 0x7f865c158510 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f865800c4d0 tx=0x7f865800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:24.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.982+0000 7f864e7fc700 1 -- 172.21.15.43:0/2764031329 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86580074e0 con 0x7f865c0cb8e0 2023-06-28T20:07:24.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.982+0000 7f864e7fc700 1 -- 172.21.15.43:0/2764031329 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8658007640 con 0x7f865c0cb8e0 2023-06-28T20:07:24.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.982+0000 7f8663c7c700 1 -- 172.21.15.43:0/2764031329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865c0c7e50 con 0x7f865c0cb8e0 2023-06-28T20:07:24.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.982+0000 7f864e7fc700 1 -- 172.21.15.43:0/2764031329 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658005a70 con 0x7f865c0cb8e0 2023-06-28T20:07:24.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.982+0000 7f8663c7c700 1 -- 172.21.15.43:0/2764031329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865c0c83a0 con 0x7f865c0cb8e0 2023-06-28T20:07:24.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.983+0000 7f8663c7c700 1 -- 172.21.15.43:0/2764031329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865c1e4c90 con 0x7f865c0cb8e0 2023-06-28T20:07:24.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.983+0000 7f864e7fc700 1 -- 172.21.15.43:0/2764031329 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8658005bd0 con 0x7f865c0cb8e0 2023-06-28T20:07:24.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.983+0000 7f864e7fc700 1 --2- 172.21.15.43:0/2764031329 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f864006ace0 0x7f864006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:24.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.984+0000 7f864e7fc700 1 -- 172.21.15.43:0/2764031329 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8658088910 con 0x7f865c0cb8e0 2023-06-28T20:07:24.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.984+0000 7f8661217700 1 --2- 172.21.15.43:0/2764031329 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f864006ace0 0x7f864006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:24.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.984+0000 7f8661217700 1 --2- 172.21.15.43:0/2764031329 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f864006ace0 0x7f864006d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f865c0cf3b0 tx=0x7f8650008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:24.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:24.988+0000 7f864e7fc700 1 -- 172.21.15.43:0/2764031329 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8658053a50 con 0x7f865c0cb8e0 2023-06-28T20:07:25.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.189+0000 7f8663c7c700 1 -- 172.21.15.43:0/2764031329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f865c0c8530 con 0x7f865c0cb8e0 2023-06-28T20:07:25.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.190+0000 7f864e7fc700 1 -- 172.21.15.43:0/2764031329 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18221 (secure 0 0 0) 0x7f8658017090 con 0x7f865c0cb8e0 2023-06-28T20:07:25.194 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:25.194 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:07:22.377248+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:07:18.624759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-28T20:07:19.760017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:07:25.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.192+0000 7f8663c7c700 1 -- 172.21.15.43:0/2764031329 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f864006ace0 msgr2=0x7f864006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:25.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.192+0000 7f8663c7c700 1 --2- 172.21.15.43:0/2764031329 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f864006ace0 0x7f864006d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f865c0cf3b0 tx=0x7f8650008030 comp rx=0 tx=0).stop 2023-06-28T20:07:25.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.192+0000 7f8663c7c700 1 -- 172.21.15.43:0/2764031329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f865c0cb8e0 msgr2=0x7f865c158510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:25.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.192+0000 7f8663c7c700 1 --2- 172.21.15.43:0/2764031329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f865c0cb8e0 0x7f865c158510 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f865800c4d0 tx=0x7f865800c890 comp rx=0 tx=0).stop 2023-06-28T20:07:25.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.192+0000 7f8663c7c700 1 -- 172.21.15.43:0/2764031329 shutdown_connections 2023-06-28T20:07:25.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.192+0000 7f8663c7c700 1 --2- 172.21.15.43:0/2764031329 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f864006ace0 0x7f864006d190 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:25.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.192+0000 7f8663c7c700 1 --2- 172.21.15.43:0/2764031329 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f865c0d10f0 0x7f865c1e3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:25.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.192+0000 7f8663c7c700 1 --2- 172.21.15.43:0/2764031329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f865c0cb8e0 0x7f865c158510 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:25.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.192+0000 7f8663c7c700 1 --2- 172.21.15.43:0/2764031329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f865c0cefc0 0x7f865c0cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:25.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.192+0000 7f8663c7c700 1 -- 172.21.15.43:0/2764031329 >> 172.21.15.43:0/2764031329 conn(0x7f865c0664b0 msgr2=0x7f865c0ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:25.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.192+0000 7f8663c7c700 1 -- 172.21.15.43:0/2764031329 shutdown_connections 2023-06-28T20:07:25.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.192+0000 7f8663c7c700 1 -- 172.21.15.43:0/2764031329 wait complete. 2023-06-28T20:07:25.207 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-06-28T20:07:25.208 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-28T20:07:25.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.330+0000 7fd09a98e700 1 Processor -- start 2023-06-28T20:07:25.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.330+0000 7fd09a98e700 1 -- start start 2023-06-28T20:07:25.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.330+0000 7fd09a98e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd09406ca80 0x7fd0940cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:25.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.330+0000 7fd09a98e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd094071410 con 0x7fd0940af060 2023-06-28T20:07:25.334 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.330+0000 7fd09a98e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0940716b0 con 0x7fd09406a600 2023-06-28T20:07:25.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.330+0000 7fd09a98e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd094071990 con 0x7fd09406ca80 2023-06-28T20:07:25.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.331+0000 7fd0937fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd09406ca80 0x7fd0940cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:25.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.331+0000 7fd093fff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd0940af060 0x7fd094069ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42102/0 (socket says 172.21.15.43:42102) 2023-06-28T20:07:25.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.331+0000 7fd093fff700 1 -- 172.21.15.43:0/3626158252 learned_addr learned my addr 172.21.15.43:0/3626158252 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:25.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.331+0000 7fd0937fe700 1 -- 172.21.15.43:0/3626158252 >> v1:172.21.15.178:6789/0 conn(0x7fd09406a600 legacy=0x7fd0940ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:25.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.331+0000 7fd0937fe700 1 -- 172.21.15.43:0/3626158252 >> v1:172.21.15.43:6789/0 conn(0x7fd0940af060 legacy=0x7fd094069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:25.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.331+0000 7fd0937fe700 1 -- 172.21.15.43:0/3626158252 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0940af430 con 0x7fd09406ca80 2023-06-28T20:07:25.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.331+0000 7fd092ffd700 1 -- 172.21.15.43:0/3626158252 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2979847583 0 0) 0x7fd094071410 con 0x7fd0940af060 2023-06-28T20:07:25.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.331+0000 7fd0937fe700 1 --2- 172.21.15.43:0/3626158252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd09406ca80 0x7fd0940cf680 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd088009840 tx=0x7fd088009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b27def640effe server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:25.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.331+0000 7fd092ffd700 1 -- 172.21.15.43:0/3626158252 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd088011830 con 0x7fd09406ca80 2023-06-28T20:07:25.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.331+0000 7fd092ffd700 1 -- 172.21.15.43:0/3626158252 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd088011990 con 0x7fd09406ca80 2023-06-28T20:07:25.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.331+0000 7fd092ffd700 1 -- 172.21.15.43:0/3626158252 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd088011d20 con 0x7fd09406ca80 2023-06-28T20:07:25.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.332+0000 7fd09a98e700 1 -- 172.21.15.43:0/3626158252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd09406ca80 msgr2=0x7fd0940cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:25.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.332+0000 7fd09a98e700 1 --2- 172.21.15.43:0/3626158252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd09406ca80 0x7fd0940cf680 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd088009840 tx=0x7fd088009b50 comp rx=0 tx=0).stop 2023-06-28T20:07:25.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.332+0000 7fd09a98e700 1 -- 172.21.15.43:0/3626158252 shutdown_connections 2023-06-28T20:07:25.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.332+0000 7fd09a98e700 1 --2- 172.21.15.43:0/3626158252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd09406ca80 0x7fd0940cf680 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:25.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.332+0000 7fd09a98e700 1 -- 172.21.15.43:0/3626158252 >> 172.21.15.43:0/3626158252 conn(0x7fd0940b1720 msgr2=0x7fd0940b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:25.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.332+0000 7fd09a98e700 1 -- 172.21.15.43:0/3626158252 shutdown_connections 2023-06-28T20:07:25.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.332+0000 7fd09a98e700 1 -- 172.21.15.43:0/3626158252 wait complete. 2023-06-28T20:07:25.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.333+0000 7fd09a98e700 1 Processor -- start 2023-06-28T20:07:25.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.333+0000 7fd09a98e700 1 -- start start 2023-06-28T20:07:25.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.334+0000 7fd09a98e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd09406a600 0x7fd094155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:25.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.334+0000 7fd09a98e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0940af060 0x7fd0941f26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:25.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.334+0000 7fd09a98e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0941f2bf0 0x7fd0941f3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:25.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.334+0000 7fd09a98e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0941598d0 con 0x7fd0941f2bf0 2023-06-28T20:07:25.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.334+0000 7fd09a98e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd094158d50 con 0x7fd0940af060 2023-06-28T20:07:25.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.334+0000 7fd09a98e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd094159030 con 0x7fd09406a600 2023-06-28T20:07:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.334+0000 7fd098f2b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0941f2bf0 0x7fd0941f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.334+0000 7fd098f2b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0941f2bf0 0x7fd0941f3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52924/0 (socket says 172.21.15.43:52924) 2023-06-28T20:07:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.334+0000 7fd098f2b700 1 -- 172.21.15.43:0/618037027 learned_addr learned my addr 172.21.15.43:0/618037027 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.334+0000 7fd093fff700 1 --2- 172.21.15.43:0/618037027 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd09406a600 0x7fd094155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.334+0000 7fd098f2b700 1 -- 172.21.15.43:0/618037027 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd09406a600 msgr2=0x7fd094155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.334+0000 7fd098f2b700 1 --2- 172.21.15.43:0/618037027 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd09406a600 0x7fd094155d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.334+0000 7fd098f2b700 1 -- 172.21.15.43:0/618037027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0940af060 msgr2=0x7fd0941f26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.334+0000 7fd098f2b700 1 --2- 172.21.15.43:0/618037027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0940af060 0x7fd0941f26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:25.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.334+0000 7fd098f2b700 1 -- 172.21.15.43:0/618037027 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0880094a0 con 0x7fd0941f2bf0 2023-06-28T20:07:25.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.334+0000 7fd098f2b700 1 --2- 172.21.15.43:0/618037027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0941f2bf0 0x7fd0941f3090 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00cbc0 tx=0x7fd08c00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:25.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.335+0000 7fd090ff9700 1 -- 172.21.15.43:0/618037027 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c003c00 con 0x7fd0941f2bf0 2023-06-28T20:07:25.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.335+0000 7fd090ff9700 1 -- 172.21.15.43:0/618037027 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08c003d60 con 0x7fd0941f2bf0 2023-06-28T20:07:25.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.335+0000 7fd09a98e700 1 -- 172.21.15.43:0/618037027 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd094159310 con 0x7fd0941f2bf0 2023-06-28T20:07:25.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.335+0000 7fd090ff9700 1 -- 172.21.15.43:0/618037027 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c00abf0 con 0x7fd0941f2bf0 2023-06-28T20:07:25.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.335+0000 7fd09a98e700 1 -- 172.21.15.43:0/618037027 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0941f35d0 con 0x7fd0941f2bf0 2023-06-28T20:07:25.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.336+0000 7fd09a98e700 1 -- 172.21.15.43:0/618037027 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0940728b0 con 0x7fd0941f2bf0 2023-06-28T20:07:25.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.336+0000 7fd090ff9700 1 -- 172.21.15.43:0/618037027 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd08c00ad50 con 0x7fd0941f2bf0 2023-06-28T20:07:25.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.336+0000 7fd090ff9700 1 --2- 172.21.15.43:0/618037027 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd08006ace0 0x7fd08006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:25.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.336+0000 7fd090ff9700 1 -- 172.21.15.43:0/618037027 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd08c089560 con 0x7fd0941f2bf0 2023-06-28T20:07:25.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.337+0000 7fd093fff700 1 --2- 172.21.15.43:0/618037027 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd08006ace0 0x7fd08006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:25.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.337+0000 7fd093fff700 1 --2- 172.21.15.43:0/618037027 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd08006ace0 0x7fd08006d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd084007fd0 tx=0x7fd084009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:25.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.341+0000 7fd090ff9700 1 -- 172.21.15.43:0/618037027 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd08c054610 con 0x7fd0941f2bf0 2023-06-28T20:07:25.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.535+0000 7fd09a98e700 1 -- 172.21.15.43:0/618037027 --> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd09406db00 con 0x7fd08006ace0 2023-06-28T20:07:25.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.544+0000 7fd090ff9700 1 -- 172.21.15.43:0/618037027 <== mgr.4101 v2:172.21.15.43:6832/406144153 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286939 (secure 0 0 0) 0x7fd0840070d0 con 0x7fd08006ace0 2023-06-28T20:07:25.550 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:25.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.547+0000 7fd09a98e700 1 -- 172.21.15.43:0/618037027 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd08006ace0 msgr2=0x7fd08006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:25.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.547+0000 7fd09a98e700 1 --2- 172.21.15.43:0/618037027 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd08006ace0 0x7fd08006d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd084007fd0 tx=0x7fd084009040 comp rx=0 tx=0).stop 2023-06-28T20:07:25.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.547+0000 7fd09a98e700 1 -- 172.21.15.43:0/618037027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0941f2bf0 msgr2=0x7fd0941f3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:25.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.547+0000 7fd09a98e700 1 --2- 172.21.15.43:0/618037027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0941f2bf0 0x7fd0941f3090 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00cbc0 tx=0x7fd08c00cf80 comp rx=0 tx=0).stop 2023-06-28T20:07:25.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.547+0000 7fd09a98e700 1 -- 172.21.15.43:0/618037027 shutdown_connections 2023-06-28T20:07:25.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.547+0000 7fd09a98e700 1 --2- 172.21.15.43:0/618037027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0941f2bf0 0x7fd0941f3090 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:25.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.547+0000 7fd09a98e700 1 --2- 172.21.15.43:0/618037027 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd08006ace0 0x7fd08006d190 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:25.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.547+0000 7fd09a98e700 1 --2- 172.21.15.43:0/618037027 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd09406a600 0x7fd094155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:25.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.547+0000 7fd09a98e700 1 --2- 172.21.15.43:0/618037027 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0940af060 0x7fd0941f26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:25.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.547+0000 7fd09a98e700 1 -- 172.21.15.43:0/618037027 >> 172.21.15.43:0/618037027 conn(0x7fd0940b1720 msgr2=0x7fd09415c180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:25.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.547+0000 7fd09a98e700 1 -- 172.21.15.43:0/618037027 shutdown_connections 2023-06-28T20:07:25.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.547+0000 7fd09a98e700 1 -- 172.21.15.43:0/618037027 wait complete. 2023-06-28T20:07:25.554 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-06-28T20:07:25.562 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-06-28T20:07:23.982651+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218460,"kb_used_data":2844,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756260,"statfs":{"total":773094113280,"available":772870410240,"internally_reserved":0,"allocated":2912256,"data_stored":1777373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.627975"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381322+0000","last_change":"2023-06-28T20:07:21.392001+0000","last_active":"2023-06-28T20:07:22.381322+0000","last_peered":"2023-06-28T20:07:22.381322+0000","last_clean":"2023-06-28T20:07:22.381322+0000","last_became_active":"2023-06-28T20:07:21.391521+0000","last_became_peered":"2023-06-28T20:07:21.391521+0000","last_unstale":"2023-06-28T20:07:22.381322+0000","last_undegraded":"2023-06-28T20:07:22.381322+0000","last_fullsized":"2023-06-28T20:07:22.381322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:30:02.751518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.386550+0000","last_change":"2023-06-28T20:07:21.381060+0000","last_active":"2023-06-28T20:07:21.386550+0000","last_peered":"2023-06-28T20:07:21.386550+0000","last_clean":"2023-06-28T20:07:21.386550+0000","last_became_active":"2023-06-28T20:07:21.380791+0000","last_became_peered":"2023-06-28T20:07:21.380791+0000","last_unstale":"2023-06-28T20:07:21.386550+0000","last_undegraded":"2023-06-28T20:07:21.386550+0000","last_fullsized":"2023-06-28T20:07:21.386550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:30:40.906018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.388311+0000","last_change":"2023-06-28T20:07:21.375652+0000","last_active":"2023-06-28T20:07:21.388311+0000","last_peered":"2023-06-28T20:07:21.388311+0000","last_clean":"2023-06-28T20:07:21.388311+0000","last_became_active":"2023-06-28T20:07:21.375192+0000","last_became_peered":"2023-06-28T20:07:21.375192+0000","last_unstale":"2023-06-28T20:07:21.388311+0000","last_undegraded":"2023-06-28T20:07:21.388311+0000","last_fullsized":"2023-06-28T20:07:21.388311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:14:35.290431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382914+0000","last_change":"2023-06-28T20:07:21.376200+0000","last_active":"2023-06-28T20:07:22.382914+0000","last_peered":"2023-06-28T20:07:22.382914+0000","last_clean":"2023-06-28T20:07:22.382914+0000","last_became_active":"2023-06-28T20:07:21.375927+0000","last_became_peered":"2023-06-28T20:07:21.375927+0000","last_unstale":"2023-06-28T20:07:22.382914+0000","last_undegraded":"2023-06-28T20:07:22.382914+0000","last_fullsized":"2023-06-28T20:07:22.382914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:04:16.061920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.395680+0000","last_change":"2023-06-28T20:07:21.394022+0000","last_active":"2023-06-28T20:07:21.395680+0000","last_peered":"2023-06-28T20:07:21.395680+0000","last_clean":"2023-06-28T20:07:21.395680+0000","last_became_active":"2023-06-28T20:07:21.393749+0000","last_became_peered":"2023-06-28T20:07:21.393749+0000","last_unstale":"2023-06-28T20:07:21.395680+0000","last_undegraded":"2023-06-28T20:07:21.395680+0000","last_fullsized":"2023-06-28T20:07:21.395680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:30:02.751518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.388662+0000","last_change":"2023-06-28T20:07:21.380285+0000","last_active":"2023-06-28T20:07:21.388662+0000","last_peered":"2023-06-28T20:07:21.388662+0000","last_clean":"2023-06-28T20:07:21.388662+0000","last_became_active":"2023-06-28T20:07:21.379978+0000","last_became_peered":"2023-06-28T20:07:21.379978+0000","last_unstale":"2023-06-28T20:07:21.388662+0000","last_undegraded":"2023-06-28T20:07:21.388662+0000","last_fullsized":"2023-06-28T20:07:21.388662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:43:18.447196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381458+0000","last_change":"2023-06-28T20:07:21.380983+0000","last_active":"2023-06-28T20:07:22.381458+0000","last_peered":"2023-06-28T20:07:22.381458+0000","last_clean":"2023-06-28T20:07:22.381458+0000","last_became_active":"2023-06-28T20:07:21.380706+0000","last_became_peered":"2023-06-28T20:07:21.380706+0000","last_unstale":"2023-06-28T20:07:22.381458+0000","last_undegraded":"2023-06-28T20:07:22.381458+0000","last_fullsized":"2023-06-28T20:07:22.381458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:43:51.244165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.385477+0000","last_change":"2023-06-28T20:07:21.376404+0000","last_active":"2023-06-28T20:07:21.385477+0000","last_peered":"2023-06-28T20:07:21.385477+0000","last_clean":"2023-06-28T20:07:21.385477+0000","last_became_active":"2023-06-28T20:07:21.376136+0000","last_became_peered":"2023-06-28T20:07:21.376136+0000","last_unstale":"2023-06-28T20:07:21.385477+0000","last_undegraded":"2023-06-28T20:07:21.385477+0000","last_fullsized":"2023-06-28T20:07:21.385477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:05:36.597881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.395486+0000","last_change":"2023-06-28T20:07:21.394452+0000","last_active":"2023-06-28T20:07:21.395486+0000","last_peered":"2023-06-28T20:07:21.395486+0000","last_clean":"2023-06-28T20:07:21.395486+0000","last_became_active":"2023-06-28T20:07:21.393190+0000","last_became_peered":"2023-06-28T20:07:21.393190+0000","last_unstale":"2023-06-28T20:07:21.395486+0000","last_undegraded":"2023-06-28T20:07:21.395486+0000","last_fullsized":"2023-06-28T20:07:21.395486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:05:29.189145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.388674+0000","last_change":"2023-06-28T20:07:21.384870+0000","last_active":"2023-06-28T20:07:21.388674+0000","last_peered":"2023-06-28T20:07:21.388674+0000","last_clean":"2023-06-28T20:07:21.388674+0000","last_became_active":"2023-06-28T20:07:21.384463+0000","last_became_peered":"2023-06-28T20:07:21.384463+0000","last_unstale":"2023-06-28T20:07:21.388674+0000","last_undegraded":"2023-06-28T20:07:21.388674+0000","last_fullsized":"2023-06-28T20:07:21.388674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:56:54.040531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.388753+0000","last_change":"2023-06-28T20:07:21.381193+0000","last_active":"2023-06-28T20:07:21.388753+0000","last_peered":"2023-06-28T20:07:21.388753+0000","last_clean":"2023-06-28T20:07:21.388753+0000","last_became_active":"2023-06-28T20:07:21.380917+0000","last_became_peered":"2023-06-28T20:07:21.380917+0000","last_unstale":"2023-06-28T20:07:21.388753+0000","last_undegraded":"2023-06-28T20:07:21.388753+0000","last_fullsized":"2023-06-28T20:07:21.388753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:34:26.678987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381377+0000","last_change":"2023-06-28T20:07:21.382341+0000","last_active":"2023-06-28T20:07:22.381377+0000","last_peered":"2023-06-28T20:07:22.381377+0000","last_clean":"2023-06-28T20:07:22.381377+0000","last_became_active":"2023-06-28T20:07:21.382032+0000","last_became_peered":"2023-06-28T20:07:21.382032+0000","last_unstale":"2023-06-28T20:07:22.381377+0000","last_undegraded":"2023-06-28T20:07:22.381377+0000","last_fullsized":"2023-06-28T20:07:22.381377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T08:00:53.732415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381426+0000","last_change":"2023-06-28T20:07:21.393006+0000","last_active":"2023-06-28T20:07:22.381426+0000","last_peered":"2023-06-28T20:07:22.381426+0000","last_clean":"2023-06-28T20:07:22.381426+0000","last_became_active":"2023-06-28T20:07:21.392710+0000","last_became_peered":"2023-06-28T20:07:21.392710+0000","last_unstale":"2023-06-28T20:07:22.381426+0000","last_undegraded":"2023-06-28T20:07:22.381426+0000","last_fullsized":"2023-06-28T20:07:22.381426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:01:21.282123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.385631+0000","last_change":"2023-06-28T20:07:21.380173+0000","last_active":"2023-06-28T20:07:21.385631+0000","last_peered":"2023-06-28T20:07:21.385631+0000","last_clean":"2023-06-28T20:07:21.385631+0000","last_became_active":"2023-06-28T20:07:21.379821+0000","last_became_peered":"2023-06-28T20:07:21.379821+0000","last_unstale":"2023-06-28T20:07:21.385631+0000","last_undegraded":"2023-06-28T20:07:21.385631+0000","last_fullsized":"2023-06-28T20:07:21.385631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:30:55.763718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381515+0000","last_change":"2023-06-28T20:07:21.381063+0000","last_active":"2023-06-28T20:07:22.381515+0000","last_peered":"2023-06-28T20:07:22.381515+0000","last_clean":"2023-06-28T20:07:22.381515+0000","last_became_active":"2023-06-28T20:07:21.380785+0000","last_became_peered":"2023-06-28T20:07:21.380785+0000","last_unstale":"2023-06-28T20:07:22.381515+0000","last_undegraded":"2023-06-28T20:07:22.381515+0000","last_fullsized":"2023-06-28T20:07:22.381515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:48:48.825002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.388719+0000","last_change":"2023-06-28T20:07:21.380326+0000","last_active":"2023-06-28T20:07:21.388719+0000","last_peered":"2023-06-28T20:07:21.388719+0000","last_clean":"2023-06-28T20:07:21.388719+0000","last_became_active":"2023-06-28T20:07:21.380043+0000","last_became_peered":"2023-06-28T20:07:21.380043+0000","last_unstale":"2023-06-28T20:07:21.388719+0000","last_undegraded":"2023-06-28T20:07:21.388719+0000","last_fullsized":"2023-06-28T20:07:21.388719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:01:21.282123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.395133+0000","last_change":"2023-06-28T20:07:21.393453+0000","last_active":"2023-06-28T20:07:21.395133+0000","last_peered":"2023-06-28T20:07:21.395133+0000","last_clean":"2023-06-28T20:07:21.395133+0000","last_became_active":"2023-06-28T20:07:21.392856+0000","last_became_peered":"2023-06-28T20:07:21.392856+0000","last_unstale":"2023-06-28T20:07:21.395133+0000","last_undegraded":"2023-06-28T20:07:21.395133+0000","last_fullsized":"2023-06-28T20:07:21.395133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:30:55.763718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.390040+0000","last_change":"2023-06-28T20:07:21.387823+0000","last_active":"2023-06-28T20:07:21.390040+0000","last_peered":"2023-06-28T20:07:21.390040+0000","last_clean":"2023-06-28T20:07:21.390040+0000","last_became_active":"2023-06-28T20:07:21.387528+0000","last_became_peered":"2023-06-28T20:07:21.387528+0000","last_unstale":"2023-06-28T20:07:21.390040+0000","last_undegraded":"2023-06-28T20:07:21.390040+0000","last_fullsized":"2023-06-28T20:07:21.390040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:16:04.779207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381385+0000","last_change":"2023-06-28T20:07:21.383174+0000","last_active":"2023-06-28T20:07:22.381385+0000","last_peered":"2023-06-28T20:07:22.381385+0000","last_clean":"2023-06-28T20:07:22.381385+0000","last_became_active":"2023-06-28T20:07:21.382865+0000","last_became_peered":"2023-06-28T20:07:21.382865+0000","last_unstale":"2023-06-28T20:07:22.381385+0000","last_undegraded":"2023-06-28T20:07:22.381385+0000","last_fullsized":"2023-06-28T20:07:22.381385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:19:03.691963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.388621+0000","last_change":"2023-06-28T20:07:21.379462+0000","last_active":"2023-06-28T20:07:21.388621+0000","last_peered":"2023-06-28T20:07:21.388621+0000","last_clean":"2023-06-28T20:07:21.388621+0000","last_became_active":"2023-06-28T20:07:21.379190+0000","last_became_peered":"2023-06-28T20:07:21.379190+0000","last_unstale":"2023-06-28T20:07:21.388621+0000","last_undegraded":"2023-06-28T20:07:21.388621+0000","last_fullsized":"2023-06-28T20:07:21.388621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:55:38.748243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381211+0000","last_change":"2023-06-28T20:07:21.392576+0000","last_active":"2023-06-28T20:07:22.381211+0000","last_peered":"2023-06-28T20:07:22.381211+0000","last_clean":"2023-06-28T20:07:22.381211+0000","last_became_active":"2023-06-28T20:07:21.392374+0000","last_became_peered":"2023-06-28T20:07:21.392374+0000","last_unstale":"2023-06-28T20:07:22.381211+0000","last_undegraded":"2023-06-28T20:07:22.381211+0000","last_fullsized":"2023-06-28T20:07:22.381211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:56:31.654337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.387803+0000","last_change":"2023-06-28T20:07:21.381495+0000","last_active":"2023-06-28T20:07:21.387803+0000","last_peered":"2023-06-28T20:07:21.387803+0000","last_clean":"2023-06-28T20:07:21.387803+0000","last_became_active":"2023-06-28T20:07:21.380960+0000","last_became_peered":"2023-06-28T20:07:21.380960+0000","last_unstale":"2023-06-28T20:07:21.387803+0000","last_undegraded":"2023-06-28T20:07:21.387803+0000","last_fullsized":"2023-06-28T20:07:21.387803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T08:00:53.732415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.395494+0000","last_change":"2023-06-28T20:07:21.393139+0000","last_active":"2023-06-28T20:07:21.395494+0000","last_peered":"2023-06-28T20:07:21.395494+0000","last_clean":"2023-06-28T20:07:21.395494+0000","last_became_active":"2023-06-28T20:07:21.392866+0000","last_became_peered":"2023-06-28T20:07:21.392866+0000","last_unstale":"2023-06-28T20:07:21.395494+0000","last_undegraded":"2023-06-28T20:07:21.395494+0000","last_fullsized":"2023-06-28T20:07:21.395494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:46:12.205864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.380893+0000","last_change":"2023-06-28T20:07:21.386233+0000","last_active":"2023-06-28T20:07:22.380893+0000","last_peered":"2023-06-28T20:07:22.380893+0000","last_clean":"2023-06-28T20:07:22.380893+0000","last_became_active":"2023-06-28T20:07:21.385374+0000","last_became_peered":"2023-06-28T20:07:21.385374+0000","last_unstale":"2023-06-28T20:07:22.380893+0000","last_undegraded":"2023-06-28T20:07:22.380893+0000","last_fullsized":"2023-06-28T20:07:22.380893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:31:11.922720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.397454+0000","last_change":"2023-06-28T20:07:21.381465+0000","last_active":"2023-06-28T20:07:21.397454+0000","last_peered":"2023-06-28T20:07:21.397454+0000","last_clean":"2023-06-28T20:07:21.397454+0000","last_became_active":"2023-06-28T20:07:21.378500+0000","last_became_peered":"2023-06-28T20:07:21.378500+0000","last_unstale":"2023-06-28T20:07:21.397454+0000","last_undegraded":"2023-06-28T20:07:21.397454+0000","last_fullsized":"2023-06-28T20:07:21.397454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:05:36.597881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.397634+0000","last_change":"2023-06-28T20:07:21.384064+0000","last_active":"2023-06-28T20:07:21.397634+0000","last_peered":"2023-06-28T20:07:21.397634+0000","last_clean":"2023-06-28T20:07:21.397634+0000","last_became_active":"2023-06-28T20:07:21.383797+0000","last_became_peered":"2023-06-28T20:07:21.383797+0000","last_unstale":"2023-06-28T20:07:21.397634+0000","last_undegraded":"2023-06-28T20:07:21.397634+0000","last_fullsized":"2023-06-28T20:07:21.397634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:56:54.040531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.388470+0000","last_change":"2023-06-28T20:07:21.382031+0000","last_active":"2023-06-28T20:07:21.388470+0000","last_peered":"2023-06-28T20:07:21.388470+0000","last_clean":"2023-06-28T20:07:21.388470+0000","last_became_active":"2023-06-28T20:07:21.381763+0000","last_became_peered":"2023-06-28T20:07:21.381763+0000","last_unstale":"2023-06-28T20:07:21.388470+0000","last_undegraded":"2023-06-28T20:07:21.388470+0000","last_fullsized":"2023-06-28T20:07:21.388470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:39:10.624775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.397890+0000","last_change":"2023-06-28T20:07:20.382468+0000","last_active":"2023-06-28T20:07:21.397890+0000","last_peered":"2023-06-28T20:07:21.397890+0000","last_clean":"2023-06-28T20:07:21.397890+0000","last_became_active":"2023-06-28T20:07:20.382185+0000","last_became_peered":"2023-06-28T20:07:20.382185+0000","last_unstale":"2023-06-28T20:07:21.397890+0000","last_undegraded":"2023-06-28T20:07:21.397890+0000","last_fullsized":"2023-06-28T20:07:21.397890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:05:45.610772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.392080+0000","last_change":"2023-06-28T20:07:21.386960+0000","last_active":"2023-06-28T20:07:21.392080+0000","last_peered":"2023-06-28T20:07:21.392080+0000","last_clean":"2023-06-28T20:07:21.392080+0000","last_became_active":"2023-06-28T20:07:21.386705+0000","last_became_peered":"2023-06-28T20:07:21.386705+0000","last_unstale":"2023-06-28T20:07:21.392080+0000","last_undegraded":"2023-06-28T20:07:21.392080+0000","last_fullsized":"2023-06-28T20:07:21.392080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:30:55.763718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.364204+0000","last_change":"2023-06-28T20:07:20.382275+0000","last_active":"2023-06-28T20:07:21.364204+0000","last_peered":"2023-06-28T20:07:21.364204+0000","last_clean":"2023-06-28T20:07:21.364204+0000","last_became_active":"2023-06-28T20:07:20.381923+0000","last_became_peered":"2023-06-28T20:07:20.381923+0000","last_unstale":"2023-06-28T20:07:21.364204+0000","last_undegraded":"2023-06-28T20:07:21.364204+0000","last_fullsized":"2023-06-28T20:07:21.364204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T08:00:52.726933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.383166+0000","last_change":"2023-06-28T20:07:21.380199+0000","last_active":"2023-06-28T20:07:21.383166+0000","last_peered":"2023-06-28T20:07:21.383166+0000","last_clean":"2023-06-28T20:07:21.383166+0000","last_became_active":"2023-06-28T20:07:21.379925+0000","last_became_peered":"2023-06-28T20:07:21.379925+0000","last_unstale":"2023-06-28T20:07:21.383166+0000","last_undegraded":"2023-06-28T20:07:21.383166+0000","last_fullsized":"2023-06-28T20:07:21.383166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:01:21.282123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.368710+0000","last_change":"2023-06-28T20:07:20.382465+0000","last_active":"2023-06-28T20:07:21.368710+0000","last_peered":"2023-06-28T20:07:21.368710+0000","last_clean":"2023-06-28T20:07:21.368710+0000","last_became_active":"2023-06-28T20:07:20.380273+0000","last_became_peered":"2023-06-28T20:07:20.380273+0000","last_unstale":"2023-06-28T20:07:21.368710+0000","last_undegraded":"2023-06-28T20:07:21.368710+0000","last_fullsized":"2023-06-28T20:07:21.368710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:56:53.035050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382428+0000","last_change":"2023-06-28T20:07:21.386881+0000","last_active":"2023-06-28T20:07:22.382428+0000","last_peered":"2023-06-28T20:07:22.382428+0000","last_clean":"2023-06-28T20:07:22.382428+0000","last_became_active":"2023-06-28T20:07:21.383395+0000","last_became_peered":"2023-06-28T20:07:21.383395+0000","last_unstale":"2023-06-28T20:07:22.382428+0000","last_undegraded":"2023-06-28T20:07:22.382428+0000","last_fullsized":"2023-06-28T20:07:22.382428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:34:26.678987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.377756+0000","last_change":"2023-06-28T20:07:20.378567+0000","last_active":"2023-06-28T20:07:21.377756+0000","last_peered":"2023-06-28T20:07:21.377756+0000","last_clean":"2023-06-28T20:07:21.377756+0000","last_became_active":"2023-06-28T20:07:20.377624+0000","last_became_peered":"2023-06-28T20:07:20.377624+0000","last_unstale":"2023-06-28T20:07:21.377756+0000","last_undegraded":"2023-06-28T20:07:21.377756+0000","last_fullsized":"2023-06-28T20:07:21.377756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:56:53.035050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.391370+0000","last_change":"2023-06-28T20:07:21.385250+0000","last_active":"2023-06-28T20:07:21.391370+0000","last_peered":"2023-06-28T20:07:21.391370+0000","last_clean":"2023-06-28T20:07:21.391370+0000","last_became_active":"2023-06-28T20:07:21.384763+0000","last_became_peered":"2023-06-28T20:07:21.384763+0000","last_unstale":"2023-06-28T20:07:21.391370+0000","last_undegraded":"2023-06-28T20:07:21.391370+0000","last_fullsized":"2023-06-28T20:07:21.391370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:05:36.597881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.371341+0000","last_change":"2023-06-28T20:07:20.376535+0000","last_active":"2023-06-28T20:07:21.371341+0000","last_peered":"2023-06-28T20:07:21.371341+0000","last_clean":"2023-06-28T20:07:21.371341+0000","last_became_active":"2023-06-28T20:07:20.376182+0000","last_became_peered":"2023-06-28T20:07:20.376182+0000","last_unstale":"2023-06-28T20:07:21.371341+0000","last_undegraded":"2023-06-28T20:07:21.371341+0000","last_fullsized":"2023-06-28T20:07:21.371341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:43:17.441714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.394657+0000","last_change":"2023-06-28T20:07:21.390529+0000","last_active":"2023-06-28T20:07:21.394657+0000","last_peered":"2023-06-28T20:07:21.394657+0000","last_clean":"2023-06-28T20:07:21.394657+0000","last_became_active":"2023-06-28T20:07:21.390249+0000","last_became_peered":"2023-06-28T20:07:21.390249+0000","last_unstale":"2023-06-28T20:07:21.394657+0000","last_undegraded":"2023-06-28T20:07:21.394657+0000","last_fullsized":"2023-06-28T20:07:21.394657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:01:21.282123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381357+0000","last_change":"2023-06-28T20:07:20.380921+0000","last_active":"2023-06-28T20:07:22.381357+0000","last_peered":"2023-06-28T20:07:22.381357+0000","last_clean":"2023-06-28T20:07:22.381357+0000","last_became_active":"2023-06-28T20:07:20.380632+0000","last_became_peered":"2023-06-28T20:07:20.380632+0000","last_unstale":"2023-06-28T20:07:22.381357+0000","last_undegraded":"2023-06-28T20:07:22.381357+0000","last_fullsized":"2023-06-28T20:07:22.381357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:03:07.390037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.396451+0000","last_change":"2023-06-28T20:07:21.379317+0000","last_active":"2023-06-28T20:07:21.396451+0000","last_peered":"2023-06-28T20:07:21.396451+0000","last_clean":"2023-06-28T20:07:21.396451+0000","last_became_active":"2023-06-28T20:07:21.379040+0000","last_became_peered":"2023-06-28T20:07:21.379040+0000","last_unstale":"2023-06-28T20:07:21.396451+0000","last_undegraded":"2023-06-28T20:07:21.396451+0000","last_fullsized":"2023-06-28T20:07:21.396451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:48:03.203326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.396786+0000","last_change":"2023-06-28T20:07:20.376485+0000","last_active":"2023-06-28T20:07:21.396786+0000","last_peered":"2023-06-28T20:07:21.396786+0000","last_clean":"2023-06-28T20:07:21.396786+0000","last_became_active":"2023-06-28T20:07:20.376240+0000","last_became_peered":"2023-06-28T20:07:20.376240+0000","last_unstale":"2023-06-28T20:07:21.396786+0000","last_undegraded":"2023-06-28T20:07:21.396786+0000","last_fullsized":"2023-06-28T20:07:21.396786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:40:58.608291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.388583+0000","last_change":"2023-06-28T20:07:21.379150+0000","last_active":"2023-06-28T20:07:21.388583+0000","last_peered":"2023-06-28T20:07:21.388583+0000","last_clean":"2023-06-28T20:07:21.388583+0000","last_became_active":"2023-06-28T20:07:21.378850+0000","last_became_peered":"2023-06-28T20:07:21.378850+0000","last_unstale":"2023-06-28T20:07:21.388583+0000","last_undegraded":"2023-06-28T20:07:21.388583+0000","last_fullsized":"2023-06-28T20:07:21.388583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:48:03.203326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381032+0000","last_change":"2023-06-28T20:07:20.378538+0000","last_active":"2023-06-28T20:07:22.381032+0000","last_peered":"2023-06-28T20:07:22.381032+0000","last_clean":"2023-06-28T20:07:22.381032+0000","last_became_active":"2023-06-28T20:07:20.378180+0000","last_became_peered":"2023-06-28T20:07:20.378180+0000","last_unstale":"2023-06-28T20:07:22.381032+0000","last_undegraded":"2023-06-28T20:07:22.381032+0000","last_fullsized":"2023-06-28T20:07:22.381032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:58:49.563930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.393234+0000","last_change":"2023-06-28T20:07:21.389643+0000","last_active":"2023-06-28T20:07:21.393234+0000","last_peered":"2023-06-28T20:07:21.393234+0000","last_clean":"2023-06-28T20:07:21.393234+0000","last_became_active":"2023-06-28T20:07:21.389163+0000","last_became_peered":"2023-06-28T20:07:21.389163+0000","last_unstale":"2023-06-28T20:07:21.393234+0000","last_undegraded":"2023-06-28T20:07:21.393234+0000","last_fullsized":"2023-06-28T20:07:21.393234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:30:40.906018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.377704+0000","last_change":"2023-06-28T20:07:20.371972+0000","last_active":"2023-06-28T20:07:21.377704+0000","last_peered":"2023-06-28T20:07:21.377704+0000","last_clean":"2023-06-28T20:07:21.377704+0000","last_became_active":"2023-06-28T20:07:20.370619+0000","last_became_peered":"2023-06-28T20:07:20.370619+0000","last_unstale":"2023-06-28T20:07:21.377704+0000","last_undegraded":"2023-06-28T20:07:21.377704+0000","last_fullsized":"2023-06-28T20:07:21.377704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:30:39.900537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.388496+0000","last_change":"2023-06-28T20:07:21.382142+0000","last_active":"2023-06-28T20:07:21.388496+0000","last_peered":"2023-06-28T20:07:21.388496+0000","last_clean":"2023-06-28T20:07:21.388496+0000","last_became_active":"2023-06-28T20:07:21.381848+0000","last_became_peered":"2023-06-28T20:07:21.381848+0000","last_unstale":"2023-06-28T20:07:21.388496+0000","last_undegraded":"2023-06-28T20:07:21.388496+0000","last_fullsized":"2023-06-28T20:07:21.388496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:05:29.189145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381309+0000","last_change":"2023-06-28T20:07:20.376749+0000","last_active":"2023-06-28T20:07:22.381309+0000","last_peered":"2023-06-28T20:07:22.381309+0000","last_clean":"2023-06-28T20:07:22.381309+0000","last_became_active":"2023-06-28T20:07:20.376465+0000","last_became_peered":"2023-06-28T20:07:20.376465+0000","last_unstale":"2023-06-28T20:07:22.381309+0000","last_undegraded":"2023-06-28T20:07:22.381309+0000","last_fullsized":"2023-06-28T20:07:22.381309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:20:39.309655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.395123+0000","last_change":"2023-06-28T20:07:21.393127+0000","last_active":"2023-06-28T20:07:21.395123+0000","last_peered":"2023-06-28T20:07:21.395123+0000","last_clean":"2023-06-28T20:07:21.395123+0000","last_became_active":"2023-06-28T20:07:21.392832+0000","last_became_peered":"2023-06-28T20:07:21.392832+0000","last_unstale":"2023-06-28T20:07:21.395123+0000","last_undegraded":"2023-06-28T20:07:21.395123+0000","last_fullsized":"2023-06-28T20:07:21.395123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:05:36.597881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381838+0000","last_change":"2023-06-28T20:07:20.380053+0000","last_active":"2023-06-28T20:07:22.381838+0000","last_peered":"2023-06-28T20:07:22.381838+0000","last_clean":"2023-06-28T20:07:22.381838+0000","last_became_active":"2023-06-28T20:07:20.379752+0000","last_became_peered":"2023-06-28T20:07:20.379752+0000","last_unstale":"2023-06-28T20:07:22.381838+0000","last_undegraded":"2023-06-28T20:07:22.381838+0000","last_fullsized":"2023-06-28T20:07:22.381838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:58:14.638684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.395185+0000","last_change":"2023-06-28T20:07:21.390679+0000","last_active":"2023-06-28T20:07:21.395185+0000","last_peered":"2023-06-28T20:07:21.395185+0000","last_clean":"2023-06-28T20:07:21.395185+0000","last_became_active":"2023-06-28T20:07:21.390332+0000","last_became_peered":"2023-06-28T20:07:21.390332+0000","last_unstale":"2023-06-28T20:07:21.395185+0000","last_undegraded":"2023-06-28T20:07:21.395185+0000","last_fullsized":"2023-06-28T20:07:21.395185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:58:15.644165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381372+0000","last_change":"2023-06-28T20:07:20.379001+0000","last_active":"2023-06-28T20:07:22.381372+0000","last_peered":"2023-06-28T20:07:22.381372+0000","last_clean":"2023-06-28T20:07:22.381372+0000","last_became_active":"2023-06-28T20:07:20.378692+0000","last_became_peered":"2023-06-28T20:07:20.378692+0000","last_unstale":"2023-06-28T20:07:22.381372+0000","last_undegraded":"2023-06-28T20:07:22.381372+0000","last_fullsized":"2023-06-28T20:07:22.381372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:50:31.576319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.385500+0000","last_change":"2023-06-28T20:07:21.377599+0000","last_active":"2023-06-28T20:07:21.385500+0000","last_peered":"2023-06-28T20:07:21.385500+0000","last_clean":"2023-06-28T20:07:21.385500+0000","last_became_active":"2023-06-28T20:07:21.377323+0000","last_became_peered":"2023-06-28T20:07:21.377323+0000","last_unstale":"2023-06-28T20:07:21.385500+0000","last_undegraded":"2023-06-28T20:07:21.385500+0000","last_fullsized":"2023-06-28T20:07:21.385500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:01:21.282123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381358+0000","last_change":"2023-06-28T20:07:20.379564+0000","last_active":"2023-06-28T20:07:22.381358+0000","last_peered":"2023-06-28T20:07:22.381358+0000","last_clean":"2023-06-28T20:07:22.381358+0000","last_became_active":"2023-06-28T20:07:20.379179+0000","last_became_peered":"2023-06-28T20:07:20.379179+0000","last_unstale":"2023-06-28T20:07:22.381358+0000","last_undegraded":"2023-06-28T20:07:22.381358+0000","last_fullsized":"2023-06-28T20:07:22.381358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:44:14.718714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.387776+0000","last_change":"2023-06-28T20:07:21.375868+0000","last_active":"2023-06-28T20:07:21.387776+0000","last_peered":"2023-06-28T20:07:21.387776+0000","last_clean":"2023-06-28T20:07:21.387776+0000","last_became_active":"2023-06-28T20:07:21.375273+0000","last_became_peered":"2023-06-28T20:07:21.375273+0000","last_unstale":"2023-06-28T20:07:21.387776+0000","last_undegraded":"2023-06-28T20:07:21.387776+0000","last_fullsized":"2023-06-28T20:07:21.387776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:05:29.189145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381183+0000","last_change":"2023-06-28T20:07:20.377080+0000","last_active":"2023-06-28T20:07:22.381183+0000","last_peered":"2023-06-28T20:07:22.381183+0000","last_clean":"2023-06-28T20:07:22.381183+0000","last_became_active":"2023-06-28T20:07:20.376266+0000","last_became_peered":"2023-06-28T20:07:20.376266+0000","last_unstale":"2023-06-28T20:07:22.381183+0000","last_undegraded":"2023-06-28T20:07:22.381183+0000","last_fullsized":"2023-06-28T20:07:22.381183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:21:31.960769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382189+0000","last_change":"2023-06-28T20:07:21.382463+0000","last_active":"2023-06-28T20:07:22.382189+0000","last_peered":"2023-06-28T20:07:22.382189+0000","last_clean":"2023-06-28T20:07:22.382189+0000","last_became_active":"2023-06-28T20:07:21.382163+0000","last_became_peered":"2023-06-28T20:07:21.382163+0000","last_unstale":"2023-06-28T20:07:22.382189+0000","last_undegraded":"2023-06-28T20:07:22.382189+0000","last_fullsized":"2023-06-28T20:07:22.382189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:44:41.666645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.369052+0000","last_change":"2023-06-28T20:07:20.380798+0000","last_active":"2023-06-28T20:07:21.369052+0000","last_peered":"2023-06-28T20:07:21.369052+0000","last_clean":"2023-06-28T20:07:21.369052+0000","last_became_active":"2023-06-28T20:07:20.380380+0000","last_became_peered":"2023-06-28T20:07:20.380380+0000","last_unstale":"2023-06-28T20:07:21.369052+0000","last_undegraded":"2023-06-28T20:07:21.369052+0000","last_fullsized":"2023-06-28T20:07:21.369052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:48:02.197845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.392246+0000","last_change":"2023-06-28T20:07:21.382097+0000","last_active":"2023-06-28T20:07:21.392246+0000","last_peered":"2023-06-28T20:07:21.392246+0000","last_clean":"2023-06-28T20:07:21.392246+0000","last_became_active":"2023-06-28T20:07:21.381678+0000","last_became_peered":"2023-06-28T20:07:21.381678+0000","last_unstale":"2023-06-28T20:07:21.392246+0000","last_undegraded":"2023-06-28T20:07:21.392246+0000","last_fullsized":"2023-06-28T20:07:21.392246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:19:03.691963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.397699+0000","last_change":"2023-06-28T20:07:20.380519+0000","last_active":"2023-06-28T20:07:21.397699+0000","last_peered":"2023-06-28T20:07:21.397699+0000","last_clean":"2023-06-28T20:07:21.397699+0000","last_became_active":"2023-06-28T20:07:20.380219+0000","last_became_peered":"2023-06-28T20:07:20.380219+0000","last_unstale":"2023-06-28T20:07:21.397699+0000","last_undegraded":"2023-06-28T20:07:21.397699+0000","last_fullsized":"2023-06-28T20:07:21.397699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:44:40.661163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.392115+0000","last_change":"2023-06-28T20:07:21.391559+0000","last_active":"2023-06-28T20:07:21.392115+0000","last_peered":"2023-06-28T20:07:21.392115+0000","last_clean":"2023-06-28T20:07:21.392115+0000","last_became_active":"2023-06-28T20:07:21.391214+0000","last_became_peered":"2023-06-28T20:07:21.391214+0000","last_unstale":"2023-06-28T20:07:21.392115+0000","last_undegraded":"2023-06-28T20:07:21.392115+0000","last_fullsized":"2023-06-28T20:07:21.392115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:07:49.232842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382144+0000","last_change":"2023-06-28T20:07:20.378412+0000","last_active":"2023-06-28T20:07:22.382144+0000","last_peered":"2023-06-28T20:07:22.382144+0000","last_clean":"2023-06-28T20:07:22.382144+0000","last_became_active":"2023-06-28T20:07:20.378102+0000","last_became_peered":"2023-06-28T20:07:20.378102+0000","last_unstale":"2023-06-28T20:07:22.382144+0000","last_undegraded":"2023-06-28T20:07:22.382144+0000","last_fullsized":"2023-06-28T20:07:22.382144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:05:45.610772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.390394+0000","last_change":"2023-06-28T20:07:21.387960+0000","last_active":"2023-06-28T20:07:21.390394+0000","last_peered":"2023-06-28T20:07:21.390394+0000","last_clean":"2023-06-28T20:07:21.390394+0000","last_became_active":"2023-06-28T20:07:21.387590+0000","last_became_peered":"2023-06-28T20:07:21.387590+0000","last_unstale":"2023-06-28T20:07:21.390394+0000","last_undegraded":"2023-06-28T20:07:21.390394+0000","last_fullsized":"2023-06-28T20:07:21.390394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:46:12.205864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.366745+0000","last_change":"2023-06-28T20:07:20.378036+0000","last_active":"2023-06-28T20:07:21.366745+0000","last_peered":"2023-06-28T20:07:21.366745+0000","last_clean":"2023-06-28T20:07:21.366745+0000","last_became_active":"2023-06-28T20:07:20.377739+0000","last_became_peered":"2023-06-28T20:07:20.377739+0000","last_unstale":"2023-06-28T20:07:21.366745+0000","last_undegraded":"2023-06-28T20:07:21.366745+0000","last_fullsized":"2023-06-28T20:07:21.366745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:56:53.035050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.388764+0000","last_change":"2023-06-28T20:07:21.379387+0000","last_active":"2023-06-28T20:07:21.388764+0000","last_peered":"2023-06-28T20:07:21.388764+0000","last_clean":"2023-06-28T20:07:21.388764+0000","last_became_active":"2023-06-28T20:07:21.379090+0000","last_became_peered":"2023-06-28T20:07:21.379090+0000","last_unstale":"2023-06-28T20:07:21.388764+0000","last_undegraded":"2023-06-28T20:07:21.388764+0000","last_fullsized":"2023-06-28T20:07:21.388764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:44:41.666645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.368360+0000","last_change":"2023-06-28T20:07:20.382672+0000","last_active":"2023-06-28T20:07:21.368360+0000","last_peered":"2023-06-28T20:07:21.368360+0000","last_clean":"2023-06-28T20:07:21.368360+0000","last_became_active":"2023-06-28T20:07:20.381085+0000","last_became_peered":"2023-06-28T20:07:20.381085+0000","last_unstale":"2023-06-28T20:07:21.368360+0000","last_undegraded":"2023-06-28T20:07:21.368360+0000","last_fullsized":"2023-06-28T20:07:21.368360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:05:28.183664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.397860+0000","last_change":"2023-06-28T20:07:21.386618+0000","last_active":"2023-06-28T20:07:21.397860+0000","last_peered":"2023-06-28T20:07:21.397860+0000","last_clean":"2023-06-28T20:07:21.397860+0000","last_became_active":"2023-06-28T20:07:21.386002+0000","last_became_peered":"2023-06-28T20:07:21.386002+0000","last_unstale":"2023-06-28T20:07:21.397860+0000","last_undegraded":"2023-06-28T20:07:21.397860+0000","last_fullsized":"2023-06-28T20:07:21.397860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:06:53.375903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.395132+0000","last_change":"2023-06-28T20:07:20.379096+0000","last_active":"2023-06-28T20:07:21.395132+0000","last_peered":"2023-06-28T20:07:21.395132+0000","last_clean":"2023-06-28T20:07:21.395132+0000","last_became_active":"2023-06-28T20:07:20.378681+0000","last_became_peered":"2023-06-28T20:07:20.378681+0000","last_unstale":"2023-06-28T20:07:21.395132+0000","last_undegraded":"2023-06-28T20:07:21.395132+0000","last_fullsized":"2023-06-28T20:07:21.395132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:06:52.370421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.369688+0000","last_change":"2023-06-28T20:07:15.815425+0000","last_active":"2023-06-28T20:07:21.369688+0000","last_peered":"2023-06-28T20:07:21.369688+0000","last_clean":"2023-06-28T20:07:21.369688+0000","last_became_active":"2023-06-28T20:07:15.815163+0000","last_became_peered":"2023-06-28T20:07:15.815163+0000","last_unstale":"2023-06-28T20:07:21.369688+0000","last_undegraded":"2023-06-28T20:07:21.369688+0000","last_fullsized":"2023-06-28T20:07:21.369688+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:04:09.994295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.391347+0000","last_change":"2023-06-28T20:07:21.385061+0000","last_active":"2023-06-28T20:07:21.391347+0000","last_peered":"2023-06-28T20:07:21.391347+0000","last_clean":"2023-06-28T20:07:21.391347+0000","last_became_active":"2023-06-28T20:07:21.384404+0000","last_became_peered":"2023-06-28T20:07:21.384404+0000","last_unstale":"2023-06-28T20:07:21.391347+0000","last_undegraded":"2023-06-28T20:07:21.391347+0000","last_fullsized":"2023-06-28T20:07:21.391347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:48:55.783926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.374509+0000","last_change":"2023-06-28T20:07:15.303675+0000","last_active":"2023-06-28T20:07:21.374509+0000","last_peered":"2023-06-28T20:07:21.374509+0000","last_clean":"2023-06-28T20:07:21.374509+0000","last_became_active":"2023-06-28T20:07:15.303350+0000","last_became_peered":"2023-06-28T20:07:15.303350+0000","last_unstale":"2023-06-28T20:07:21.374509+0000","last_undegraded":"2023-06-28T20:07:21.374509+0000","last_fullsized":"2023-06-28T20:07:21.374509+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:34:20.611362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.378021+0000","last_change":"2023-06-28T20:07:20.381457+0000","last_active":"2023-06-28T20:07:21.378021+0000","last_peered":"2023-06-28T20:07:21.378021+0000","last_clean":"2023-06-28T20:07:21.378021+0000","last_became_active":"2023-06-28T20:07:20.381235+0000","last_became_peered":"2023-06-28T20:07:20.381235+0000","last_unstale":"2023-06-28T20:07:21.378021+0000","last_undegraded":"2023-06-28T20:07:21.378021+0000","last_fullsized":"2023-06-28T20:07:21.378021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:39:09.619294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.388793+0000","last_change":"2023-06-28T20:07:21.383647+0000","last_active":"2023-06-28T20:07:21.388793+0000","last_peered":"2023-06-28T20:07:21.388793+0000","last_clean":"2023-06-28T20:07:21.388793+0000","last_became_active":"2023-06-28T20:07:21.383392+0000","last_became_peered":"2023-06-28T20:07:21.383392+0000","last_unstale":"2023-06-28T20:07:21.388793+0000","last_undegraded":"2023-06-28T20:07:21.388793+0000","last_fullsized":"2023-06-28T20:07:21.388793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:05:36.597881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.364257+0000","last_change":"2023-06-28T20:07:15.303565+0000","last_active":"2023-06-28T20:07:21.364257+0000","last_peered":"2023-06-28T20:07:21.364257+0000","last_clean":"2023-06-28T20:07:21.364257+0000","last_became_active":"2023-06-28T20:07:15.303200+0000","last_became_peered":"2023-06-28T20:07:15.303200+0000","last_unstale":"2023-06-28T20:07:21.364257+0000","last_undegraded":"2023-06-28T20:07:21.364257+0000","last_fullsized":"2023-06-28T20:07:21.364257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:01:15.214498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":169,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381052+0000","last_change":"2023-06-28T20:07:15.304921+0000","last_active":"2023-06-28T20:07:22.381052+0000","last_peered":"2023-06-28T20:07:22.381052+0000","last_clean":"2023-06-28T20:07:22.381052+0000","last_became_active":"2023-06-28T20:07:15.303959+0000","last_became_peered":"2023-06-28T20:07:15.303959+0000","last_unstale":"2023-06-28T20:07:22.381052+0000","last_undegraded":"2023-06-28T20:07:22.381052+0000","last_fullsized":"2023-06-28T20:07:22.381052+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:44:42.249408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.375146+0000","last_change":"2023-06-28T20:07:20.379375+0000","last_active":"2023-06-28T20:07:21.375146+0000","last_peered":"2023-06-28T20:07:21.375146+0000","last_clean":"2023-06-28T20:07:21.375146+0000","last_became_active":"2023-06-28T20:07:20.378992+0000","last_became_peered":"2023-06-28T20:07:20.378992+0000","last_unstale":"2023-06-28T20:07:21.375146+0000","last_undegraded":"2023-06-28T20:07:21.375146+0000","last_fullsized":"2023-06-28T20:07:21.375146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:48:54.778445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.393013+0000","last_change":"2023-06-28T20:07:21.383270+0000","last_active":"2023-06-28T20:07:21.393013+0000","last_peered":"2023-06-28T20:07:21.393013+0000","last_clean":"2023-06-28T20:07:21.393013+0000","last_became_active":"2023-06-28T20:07:21.382983+0000","last_became_peered":"2023-06-28T20:07:21.382983+0000","last_unstale":"2023-06-28T20:07:21.393013+0000","last_undegraded":"2023-06-28T20:07:21.393013+0000","last_fullsized":"2023-06-28T20:07:21.393013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:48:03.203326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.380861+0000","last_change":"2023-06-28T20:07:15.899048+0000","last_active":"2023-06-28T20:07:22.380861+0000","last_peered":"2023-06-28T20:07:22.380861+0000","last_clean":"2023-06-28T20:07:22.380861+0000","last_became_active":"2023-06-28T20:07:15.898766+0000","last_became_peered":"2023-06-28T20:07:15.898766+0000","last_unstale":"2023-06-28T20:07:22.380861+0000","last_undegraded":"2023-06-28T20:07:22.380861+0000","last_fullsized":"2023-06-28T20:07:22.380861+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:27:00.869776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.364255+0000","last_change":"2023-06-28T20:07:20.369864+0000","last_active":"2023-06-28T20:07:21.364255+0000","last_peered":"2023-06-28T20:07:21.364255+0000","last_clean":"2023-06-28T20:07:21.364255+0000","last_became_active":"2023-06-28T20:07:20.369542+0000","last_became_peered":"2023-06-28T20:07:20.369542+0000","last_unstale":"2023-06-28T20:07:21.364255+0000","last_undegraded":"2023-06-28T20:07:21.364255+0000","last_fullsized":"2023-06-28T20:07:21.364255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:48:54.778445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.397567+0000","last_change":"2023-06-28T20:07:21.383019+0000","last_active":"2023-06-28T20:07:21.397567+0000","last_peered":"2023-06-28T20:07:21.397567+0000","last_clean":"2023-06-28T20:07:21.397567+0000","last_became_active":"2023-06-28T20:07:21.382756+0000","last_became_peered":"2023-06-28T20:07:21.382756+0000","last_unstale":"2023-06-28T20:07:21.397567+0000","last_undegraded":"2023-06-28T20:07:21.397567+0000","last_fullsized":"2023-06-28T20:07:21.397567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:05:29.189145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.380789+0000","last_change":"2023-06-28T20:07:15.899229+0000","last_active":"2023-06-28T20:07:22.380789+0000","last_peered":"2023-06-28T20:07:22.380789+0000","last_clean":"2023-06-28T20:07:22.380789+0000","last_became_active":"2023-06-28T20:07:15.898708+0000","last_became_peered":"2023-06-28T20:07:15.898708+0000","last_unstale":"2023-06-28T20:07:22.380789+0000","last_undegraded":"2023-06-28T20:07:22.380789+0000","last_fullsized":"2023-06-28T20:07:22.380789+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:28:19.249906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.369286+0000","last_change":"2023-06-28T20:07:20.381183+0000","last_active":"2023-06-28T20:07:21.369286+0000","last_peered":"2023-06-28T20:07:21.369286+0000","last_clean":"2023-06-28T20:07:21.369286+0000","last_became_active":"2023-06-28T20:07:20.380859+0000","last_became_peered":"2023-06-28T20:07:20.380859+0000","last_unstale":"2023-06-28T20:07:21.369286+0000","last_undegraded":"2023-06-28T20:07:21.369286+0000","last_fullsized":"2023-06-28T20:07:21.369286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:39:09.619294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382351+0000","last_change":"2023-06-28T20:07:21.382900+0000","last_active":"2023-06-28T20:07:22.382351+0000","last_peered":"2023-06-28T20:07:22.382351+0000","last_clean":"2023-06-28T20:07:22.382351+0000","last_became_active":"2023-06-28T20:07:21.382384+0000","last_became_peered":"2023-06-28T20:07:21.382384+0000","last_unstale":"2023-06-28T20:07:22.382351+0000","last_undegraded":"2023-06-28T20:07:22.382351+0000","last_fullsized":"2023-06-28T20:07:22.382351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:16:04.779207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.368676+0000","last_change":"2023-06-28T20:07:15.898966+0000","last_active":"2023-06-28T20:07:21.368676+0000","last_peered":"2023-06-28T20:07:21.368676+0000","last_clean":"2023-06-28T20:07:21.368676+0000","last_became_active":"2023-06-28T20:07:15.898704+0000","last_became_peered":"2023-06-28T20:07:15.898704+0000","last_unstale":"2023-06-28T20:07:21.368676+0000","last_undegraded":"2023-06-28T20:07:21.368676+0000","last_fullsized":"2023-06-28T20:07:21.368676+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T08:00:47.664790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.367218+0000","last_change":"2023-06-28T20:07:20.378848+0000","last_active":"2023-06-28T20:07:21.367218+0000","last_peered":"2023-06-28T20:07:21.367218+0000","last_clean":"2023-06-28T20:07:21.367218+0000","last_became_active":"2023-06-28T20:07:20.378438+0000","last_became_peered":"2023-06-28T20:07:20.378438+0000","last_unstale":"2023-06-28T20:07:21.367218+0000","last_undegraded":"2023-06-28T20:07:21.367218+0000","last_fullsized":"2023-06-28T20:07:21.367218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:58:14.638684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.397694+0000","last_change":"2023-06-28T20:07:21.384171+0000","last_active":"2023-06-28T20:07:21.397694+0000","last_peered":"2023-06-28T20:07:21.397694+0000","last_clean":"2023-06-28T20:07:21.397694+0000","last_became_active":"2023-06-28T20:07:21.383867+0000","last_became_peered":"2023-06-28T20:07:21.383867+0000","last_unstale":"2023-06-28T20:07:21.397694+0000","last_undegraded":"2023-06-28T20:07:21.397694+0000","last_fullsized":"2023-06-28T20:07:21.397694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:39:10.624775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382412+0000","last_change":"2023-06-28T20:07:15.899207+0000","last_active":"2023-06-28T20:07:22.382412+0000","last_peered":"2023-06-28T20:07:22.382412+0000","last_clean":"2023-06-28T20:07:22.382412+0000","last_became_active":"2023-06-28T20:07:15.898908+0000","last_became_peered":"2023-06-28T20:07:15.898908+0000","last_unstale":"2023-06-28T20:07:22.382412+0000","last_undegraded":"2023-06-28T20:07:22.382412+0000","last_fullsized":"2023-06-28T20:07:22.382412+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:14:29.222806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.365573+0000","last_change":"2023-06-28T20:07:20.377530+0000","last_active":"2023-06-28T20:07:21.365573+0000","last_peered":"2023-06-28T20:07:21.365573+0000","last_clean":"2023-06-28T20:07:21.365573+0000","last_became_active":"2023-06-28T20:07:20.377243+0000","last_became_peered":"2023-06-28T20:07:20.377243+0000","last_unstale":"2023-06-28T20:07:21.365573+0000","last_undegraded":"2023-06-28T20:07:21.365573+0000","last_fullsized":"2023-06-28T20:07:21.365573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:58:14.638684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.392290+0000","last_change":"2023-06-28T20:07:21.383886+0000","last_active":"2023-06-28T20:07:21.392290+0000","last_peered":"2023-06-28T20:07:21.392290+0000","last_clean":"2023-06-28T20:07:21.392290+0000","last_became_active":"2023-06-28T20:07:21.383572+0000","last_became_peered":"2023-06-28T20:07:21.383572+0000","last_unstale":"2023-06-28T20:07:21.392290+0000","last_undegraded":"2023-06-28T20:07:21.392290+0000","last_fullsized":"2023-06-28T20:07:21.392290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:06:01.974529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382342+0000","last_change":"2023-06-28T20:07:15.303873+0000","last_active":"2023-06-28T20:07:22.382342+0000","last_peered":"2023-06-28T20:07:22.382342+0000","last_clean":"2023-06-28T20:07:22.382342+0000","last_became_active":"2023-06-28T20:07:15.303457+0000","last_became_peered":"2023-06-28T20:07:15.303457+0000","last_unstale":"2023-06-28T20:07:22.382342+0000","last_undegraded":"2023-06-28T20:07:22.382342+0000","last_fullsized":"2023-06-28T20:07:22.382342+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:46:06.138239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.370912+0000","last_change":"2023-06-28T20:07:20.381529+0000","last_active":"2023-06-28T20:07:21.370912+0000","last_peered":"2023-06-28T20:07:21.370912+0000","last_clean":"2023-06-28T20:07:21.370912+0000","last_became_active":"2023-06-28T20:07:20.381191+0000","last_became_peered":"2023-06-28T20:07:20.381191+0000","last_unstale":"2023-06-28T20:07:21.370912+0000","last_undegraded":"2023-06-28T20:07:21.370912+0000","last_fullsized":"2023-06-28T20:07:21.370912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:58:14.638684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.396345+0000","last_change":"2023-06-28T20:07:21.377519+0000","last_active":"2023-06-28T20:07:21.396345+0000","last_peered":"2023-06-28T20:07:21.396345+0000","last_clean":"2023-06-28T20:07:21.396345+0000","last_became_active":"2023-06-28T20:07:21.377225+0000","last_became_peered":"2023-06-28T20:07:21.377225+0000","last_unstale":"2023-06-28T20:07:21.396345+0000","last_undegraded":"2023-06-28T20:07:21.396345+0000","last_fullsized":"2023-06-28T20:07:21.396345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:30:40.906018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.394157+0000","last_change":"2023-06-28T20:07:20.382678+0000","last_active":"2023-06-28T20:07:21.394157+0000","last_peered":"2023-06-28T20:07:21.394157+0000","last_clean":"2023-06-28T20:07:21.394157+0000","last_became_active":"2023-06-28T20:07:20.382361+0000","last_became_peered":"2023-06-28T20:07:20.382361+0000","last_unstale":"2023-06-28T20:07:21.394157+0000","last_undegraded":"2023-06-28T20:07:21.394157+0000","last_fullsized":"2023-06-28T20:07:21.394157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:06:52.370421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381381+0000","last_change":"2023-06-28T20:07:21.397232+0000","last_active":"2023-06-28T20:07:22.381381+0000","last_peered":"2023-06-28T20:07:22.381381+0000","last_clean":"2023-06-28T20:07:22.381381+0000","last_became_active":"2023-06-28T20:07:21.396862+0000","last_became_peered":"2023-06-28T20:07:21.396862+0000","last_unstale":"2023-06-28T20:07:22.381381+0000","last_undegraded":"2023-06-28T20:07:22.381381+0000","last_fullsized":"2023-06-28T20:07:22.381381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:48:55.783926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.381592+0000","last_change":"2023-06-28T20:07:20.380628+0000","last_active":"2023-06-28T20:07:21.381592+0000","last_peered":"2023-06-28T20:07:21.381592+0000","last_clean":"2023-06-28T20:07:21.381592+0000","last_became_active":"2023-06-28T20:07:20.380033+0000","last_became_peered":"2023-06-28T20:07:20.380033+0000","last_unstale":"2023-06-28T20:07:21.381592+0000","last_undegraded":"2023-06-28T20:07:21.381592+0000","last_fullsized":"2023-06-28T20:07:21.381592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:48:54.778445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.379245+0000","last_change":"2023-06-28T20:07:21.374666+0000","last_active":"2023-06-28T20:07:21.379245+0000","last_peered":"2023-06-28T20:07:21.379245+0000","last_clean":"2023-06-28T20:07:21.379245+0000","last_became_active":"2023-06-28T20:07:21.374330+0000","last_became_peered":"2023-06-28T20:07:21.374330+0000","last_unstale":"2023-06-28T20:07:21.379245+0000","last_undegraded":"2023-06-28T20:07:21.379245+0000","last_fullsized":"2023-06-28T20:07:21.379245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:39:10.624775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.381587+0000","last_change":"2023-06-28T20:07:20.378725+0000","last_active":"2023-06-28T20:07:21.381587+0000","last_peered":"2023-06-28T20:07:21.381587+0000","last_clean":"2023-06-28T20:07:21.381587+0000","last_became_active":"2023-06-28T20:07:20.378445+0000","last_became_peered":"2023-06-28T20:07:20.378445+0000","last_unstale":"2023-06-28T20:07:21.381587+0000","last_undegraded":"2023-06-28T20:07:21.381587+0000","last_fullsized":"2023-06-28T20:07:21.381587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:06:52.370421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.378602+0000","last_change":"2023-06-28T20:07:21.374603+0000","last_active":"2023-06-28T20:07:21.378602+0000","last_peered":"2023-06-28T20:07:21.378602+0000","last_clean":"2023-06-28T20:07:21.378602+0000","last_became_active":"2023-06-28T20:07:21.374276+0000","last_became_peered":"2023-06-28T20:07:21.374276+0000","last_unstale":"2023-06-28T20:07:21.378602+0000","last_undegraded":"2023-06-28T20:07:21.378602+0000","last_fullsized":"2023-06-28T20:07:21.378602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:30:40.906018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.393258+0000","last_change":"2023-06-28T20:07:20.382290+0000","last_active":"2023-06-28T20:07:21.393258+0000","last_peered":"2023-06-28T20:07:21.393258+0000","last_clean":"2023-06-28T20:07:21.393258+0000","last_became_active":"2023-06-28T20:07:20.381991+0000","last_became_peered":"2023-06-28T20:07:20.381991+0000","last_unstale":"2023-06-28T20:07:21.393258+0000","last_undegraded":"2023-06-28T20:07:21.393258+0000","last_fullsized":"2023-06-28T20:07:21.393258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:48:02.197845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.394677+0000","last_change":"2023-06-28T20:07:21.391299+0000","last_active":"2023-06-28T20:07:21.394677+0000","last_peered":"2023-06-28T20:07:21.394677+0000","last_clean":"2023-06-28T20:07:21.394677+0000","last_became_active":"2023-06-28T20:07:21.390957+0000","last_became_peered":"2023-06-28T20:07:21.390957+0000","last_unstale":"2023-06-28T20:07:21.394677+0000","last_undegraded":"2023-06-28T20:07:21.394677+0000","last_fullsized":"2023-06-28T20:07:21.394677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:05:46.616253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.392610+0000","last_change":"2023-06-28T20:07:20.378108+0000","last_active":"2023-06-28T20:07:21.392610+0000","last_peered":"2023-06-28T20:07:21.392610+0000","last_clean":"2023-06-28T20:07:21.392610+0000","last_became_active":"2023-06-28T20:07:20.377812+0000","last_became_peered":"2023-06-28T20:07:20.377812+0000","last_unstale":"2023-06-28T20:07:21.392610+0000","last_undegraded":"2023-06-28T20:07:21.392610+0000","last_fullsized":"2023-06-28T20:07:21.392610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:58:14.638684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.389928+0000","last_change":"2023-06-28T20:07:21.384739+0000","last_active":"2023-06-28T20:07:21.389928+0000","last_peered":"2023-06-28T20:07:21.389928+0000","last_clean":"2023-06-28T20:07:21.389928+0000","last_became_active":"2023-06-28T20:07:21.384136+0000","last_became_peered":"2023-06-28T20:07:21.384136+0000","last_unstale":"2023-06-28T20:07:21.389928+0000","last_undegraded":"2023-06-28T20:07:21.389928+0000","last_fullsized":"2023-06-28T20:07:21.389928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:30:55.763718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382228+0000","last_change":"2023-06-28T20:07:21.386188+0000","last_active":"2023-06-28T20:07:22.382228+0000","last_peered":"2023-06-28T20:07:22.382228+0000","last_clean":"2023-06-28T20:07:22.382228+0000","last_became_active":"2023-06-28T20:07:21.385171+0000","last_became_peered":"2023-06-28T20:07:21.385171+0000","last_unstale":"2023-06-28T20:07:22.382228+0000","last_undegraded":"2023-06-28T20:07:22.382228+0000","last_fullsized":"2023-06-28T20:07:22.382228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:43:18.447196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.402926+0000","last_change":"2023-06-28T20:07:21.402344+0000","last_active":"2023-06-28T20:07:21.402926+0000","last_peered":"2023-06-28T20:07:21.402926+0000","last_clean":"2023-06-28T20:07:21.402926+0000","last_became_active":"2023-06-28T20:07:21.402005+0000","last_became_peered":"2023-06-28T20:07:21.402005+0000","last_unstale":"2023-06-28T20:07:21.402926+0000","last_undegraded":"2023-06-28T20:07:21.402926+0000","last_fullsized":"2023-06-28T20:07:21.402926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:16:04.779207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.389900+0000","last_change":"2023-06-28T20:07:21.384643+0000","last_active":"2023-06-28T20:07:21.389900+0000","last_peered":"2023-06-28T20:07:21.389900+0000","last_clean":"2023-06-28T20:07:21.389900+0000","last_became_active":"2023-06-28T20:07:21.384232+0000","last_became_peered":"2023-06-28T20:07:21.384232+0000","last_unstale":"2023-06-28T20:07:21.389900+0000","last_undegraded":"2023-06-28T20:07:21.389900+0000","last_fullsized":"2023-06-28T20:07:21.389900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:06:53.375903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.394714+0000","last_change":"2023-06-28T20:07:21.391594+0000","last_active":"2023-06-28T20:07:21.394714+0000","last_peered":"2023-06-28T20:07:21.394714+0000","last_clean":"2023-06-28T20:07:21.394714+0000","last_became_active":"2023-06-28T20:07:21.391222+0000","last_became_peered":"2023-06-28T20:07:21.391222+0000","last_unstale":"2023-06-28T20:07:21.394714+0000","last_undegraded":"2023-06-28T20:07:21.394714+0000","last_fullsized":"2023-06-28T20:07:21.394714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:43:18.447196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.396508+0000","last_change":"2023-06-28T20:07:21.378640+0000","last_active":"2023-06-28T20:07:21.396508+0000","last_peered":"2023-06-28T20:07:21.396508+0000","last_clean":"2023-06-28T20:07:21.396508+0000","last_became_active":"2023-06-28T20:07:21.378345+0000","last_became_peered":"2023-06-28T20:07:21.378345+0000","last_unstale":"2023-06-28T20:07:21.396508+0000","last_undegraded":"2023-06-28T20:07:21.396508+0000","last_fullsized":"2023-06-28T20:07:21.396508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:30:55.763718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":839680,"data_stored":671247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":79099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":798720,"data_stored":662814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":72067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":72446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":208896,"data_stored":72536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":225280,"data_stored":74066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":212992,"data_stored":73098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-28T20:07:25.569 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-28T20:07:25.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.724+0000 7f65605d8700 1 Processor -- start 2023-06-28T20:07:25.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.724+0000 7f65605d8700 1 -- start start 2023-06-28T20:07:25.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.725+0000 7f65605d8700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f65580cb1f0 0x7f655815b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:25.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.725+0000 7f65605d8700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f65580cb5c0 0x7f65580cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:25.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.725+0000 7f65605d8700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f655815e250 con 0x7f655806b230 2023-06-28T20:07:25.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.725+0000 7f65605d8700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f655815e040 con 0x7f65580cb5c0 2023-06-28T20:07:25.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.725+0000 7f65605d8700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f655815f010 con 0x7f65580cb1f0 2023-06-28T20:07:25.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.725+0000 7f655db73700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f65580cb5c0 0x7f65580cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:25.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.725+0000 7f655eb75700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f655806b230 0x7f6558069ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42116/0 (socket says 172.21.15.43:42116) 2023-06-28T20:07:25.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.725+0000 7f655db73700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f65580cb5c0 0x7f65580cbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52930/0 (socket says 172.21.15.43:52930) 2023-06-28T20:07:25.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.725+0000 7f655eb75700 1 -- 172.21.15.43:0/406227124 learned_addr learned my addr 172.21.15.43:0/406227124 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:25.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.725+0000 7f655e374700 1 --2- 172.21.15.43:0/406227124 >> v2:172.21.15.178:3300/0 conn(0x7f65580cb1f0 0x7f655815b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:25.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.725+0000 7f655db73700 1 -- 172.21.15.43:0/406227124 >> v2:172.21.15.178:3300/0 conn(0x7f65580cb1f0 msgr2=0x7f655815b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:25.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.725+0000 7f655db73700 1 --2- 172.21.15.43:0/406227124 >> v2:172.21.15.178:3300/0 conn(0x7f65580cb1f0 0x7f655815b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:25.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.725+0000 7f655db73700 1 -- 172.21.15.43:0/406227124 >> v1:172.21.15.43:6789/0 conn(0x7f655806b230 legacy=0x7f6558069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:25.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.725+0000 7f655d372700 1 -- 172.21.15.43:0/406227124 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665692257 0 0) 0x7f655815e250 con 0x7f655806b230 2023-06-28T20:07:25.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.725+0000 7f655db73700 1 -- 172.21.15.43:0/406227124 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655815e530 con 0x7f65580cb5c0 2023-06-28T20:07:25.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.726+0000 7f655db73700 1 --2- 172.21.15.43:0/406227124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65580cb5c0 0x7f65580cbcc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f65580cce30 tx=0x7f6554009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f1c3f0ef2a5232 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:25.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.726+0000 7f655d372700 1 -- 172.21.15.43:0/406227124 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6554011830 con 0x7f65580cb5c0 2023-06-28T20:07:25.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.726+0000 7f655d372700 1 -- 172.21.15.43:0/406227124 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6554011990 con 0x7f65580cb5c0 2023-06-28T20:07:25.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.726+0000 7f655d372700 1 -- 172.21.15.43:0/406227124 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6554011d20 con 0x7f65580cb5c0 2023-06-28T20:07:25.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.726+0000 7f65605d8700 1 -- 172.21.15.43:0/406227124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65580cb5c0 msgr2=0x7f65580cbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:25.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.726+0000 7f65605d8700 1 --2- 172.21.15.43:0/406227124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65580cb5c0 0x7f65580cbcc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f65580cce30 tx=0x7f6554009b20 comp rx=0 tx=0).stop 2023-06-28T20:07:25.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.726+0000 7f65605d8700 1 -- 172.21.15.43:0/406227124 shutdown_connections 2023-06-28T20:07:25.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.726+0000 7f65605d8700 1 --2- 172.21.15.43:0/406227124 >> v2:172.21.15.178:3300/0 conn(0x7f65580cb1f0 0x7f655815b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:25.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.726+0000 7f65605d8700 1 --2- 172.21.15.43:0/406227124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65580cb5c0 0x7f65580cbcc0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:25.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.726+0000 7f65605d8700 1 -- 172.21.15.43:0/406227124 >> 172.21.15.43:0/406227124 conn(0x7f655806c170 msgr2=0x7f655806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:25.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.726+0000 7f65605d8700 1 -- 172.21.15.43:0/406227124 shutdown_connections 2023-06-28T20:07:25.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.726+0000 7f65605d8700 1 -- 172.21.15.43:0/406227124 wait complete. 2023-06-28T20:07:25.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.728+0000 7f65605d8700 1 Processor -- start 2023-06-28T20:07:25.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.728+0000 7f65605d8700 1 -- start start 2023-06-28T20:07:25.738 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.728+0000 7f65605d8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f655806b230 0x7f655814d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:25.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.728+0000 7f65605d8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65580cb1f0 0x7f655815ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:25.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.728+0000 7f65605d8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65580cb5c0 0x7f65581f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:25.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.728+0000 7f65605d8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65580c5310 con 0x7f655806b230 2023-06-28T20:07:25.739 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.728+0000 7f65605d8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655815d360 con 0x7f65580cb1f0 2023-06-28T20:07:25.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.728+0000 7f65605d8700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65580cd710 con 0x7f65580cb5c0 2023-06-28T20:07:25.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.729+0000 7f655e374700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f655806b230 0x7f655814d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:25.740 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.729+0000 7f655e374700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f655806b230 0x7f655814d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52936/0 (socket says 172.21.15.43:52936) 2023-06-28T20:07:25.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.729+0000 7f655e374700 1 -- 172.21.15.43:0/1851617889 learned_addr learned my addr 172.21.15.43:0/1851617889 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:25.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.729+0000 7f655eb75700 1 --2- 172.21.15.43:0/1851617889 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65580cb5c0 0x7f65581f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:25.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.729+0000 7f655db73700 1 --2- 172.21.15.43:0/1851617889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65580cb1f0 0x7f655815ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:25.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.729+0000 7f655e374700 1 -- 172.21.15.43:0/1851617889 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65580cb5c0 msgr2=0x7f65581f4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:25.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.729+0000 7f655e374700 1 --2- 172.21.15.43:0/1851617889 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65580cb5c0 0x7f65581f4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:25.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.729+0000 7f655e374700 1 -- 172.21.15.43:0/1851617889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65580cb1f0 msgr2=0x7f655815ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:25.742 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.729+0000 7f655e374700 1 --2- 172.21.15.43:0/1851617889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65580cb1f0 0x7f655815ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:25.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.729+0000 7f655e374700 1 -- 172.21.15.43:0/1851617889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65540094a0 con 0x7f655806b230 2023-06-28T20:07:25.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.729+0000 7f655e374700 1 --2- 172.21.15.43:0/1851617889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f655806b230 0x7f655814d5d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f654800c820 tx=0x7f654800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:25.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.729+0000 7f654effd700 1 -- 172.21.15.43:0/1851617889 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6548011800 con 0x7f655806b230 2023-06-28T20:07:25.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.729+0000 7f654effd700 1 -- 172.21.15.43:0/1851617889 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6548011960 con 0x7f655806b230 2023-06-28T20:07:25.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.729+0000 7f65605d8700 1 -- 172.21.15.43:0/1851617889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f655814df70 con 0x7f655806b230 2023-06-28T20:07:25.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.729+0000 7f654effd700 1 -- 172.21.15.43:0/1851617889 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6548011d10 con 0x7f655806b230 2023-06-28T20:07:25.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.729+0000 7f65605d8700 1 -- 172.21.15.43:0/1851617889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65580cd0c0 con 0x7f655806b230 2023-06-28T20:07:25.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.730+0000 7f65605d8700 1 -- 172.21.15.43:0/1851617889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65580728b0 con 0x7f655806b230 2023-06-28T20:07:25.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.731+0000 7f654effd700 1 -- 172.21.15.43:0/1851617889 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6548018360 con 0x7f655806b230 2023-06-28T20:07:25.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.731+0000 7f654effd700 1 --2- 172.21.15.43:0/1851617889 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f653c06ace0 0x7f653c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:25.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.731+0000 7f654effd700 1 -- 172.21.15.43:0/1851617889 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f65480893c0 con 0x7f655806b230 2023-06-28T20:07:25.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.731+0000 7f655db73700 1 --2- 172.21.15.43:0/1851617889 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f653c06ace0 0x7f653c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:25.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.732+0000 7f655db73700 1 --2- 172.21.15.43:0/1851617889 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f653c06ace0 0x7f653c06d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f65540090b0 tx=0x7f655401c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:25.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.736+0000 7f654effd700 1 -- 172.21.15.43:0/1851617889 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6548054500 con 0x7f655806b230 2023-06-28T20:07:25.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.931+0000 7f65605d8700 1 -- 172.21.15.43:0/1851617889 --> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f655815eaf0 con 0x7f653c06ace0 2023-06-28T20:07:25.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.939+0000 7f654effd700 1 -- 172.21.15.43:0/1851617889 <== mgr.4101 v2:172.21.15.43:6832/406144153 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286939 (secure 0 0 0) 0x7f6554011c80 con 0x7f653c06ace0 2023-06-28T20:07:25.947 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:25.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.942+0000 7f65605d8700 1 -- 172.21.15.43:0/1851617889 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f653c06ace0 msgr2=0x7f653c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:25.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.942+0000 7f65605d8700 1 --2- 172.21.15.43:0/1851617889 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f653c06ace0 0x7f653c06d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f65540090b0 tx=0x7f655401c040 comp rx=0 tx=0).stop 2023-06-28T20:07:25.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.942+0000 7f65605d8700 1 -- 172.21.15.43:0/1851617889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f655806b230 msgr2=0x7f655814d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:25.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.942+0000 7f65605d8700 1 --2- 172.21.15.43:0/1851617889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f655806b230 0x7f655814d5d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f654800c820 tx=0x7f654800cbe0 comp rx=0 tx=0).stop 2023-06-28T20:07:25.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.942+0000 7f65605d8700 1 -- 172.21.15.43:0/1851617889 shutdown_connections 2023-06-28T20:07:25.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.942+0000 7f65605d8700 1 --2- 172.21.15.43:0/1851617889 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f653c06ace0 0x7f653c06d190 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:25.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.942+0000 7f65605d8700 1 --2- 172.21.15.43:0/1851617889 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f65580cb5c0 0x7f65581f4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:25.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.942+0000 7f65605d8700 1 --2- 172.21.15.43:0/1851617889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f655806b230 0x7f655814d5d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:25.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.942+0000 7f65605d8700 1 --2- 172.21.15.43:0/1851617889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65580cb1f0 0x7f655815ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:25.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.942+0000 7f65605d8700 1 -- 172.21.15.43:0/1851617889 >> 172.21.15.43:0/1851617889 conn(0x7f655806c170 msgr2=0x7f65580afe30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:25.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.943+0000 7f65605d8700 1 -- 172.21.15.43:0/1851617889 shutdown_connections 2023-06-28T20:07:25.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:25.943+0000 7f65605d8700 1 -- 172.21.15.43:0/1851617889 wait complete. 2023-06-28T20:07:25.952 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-06-28T20:07:25.958 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-06-28T20:07:23.982651+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218460,"kb_used_data":2844,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756260,"statfs":{"total":773094113280,"available":772870410240,"internally_reserved":0,"allocated":2912256,"data_stored":1777373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.627975"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381322+0000","last_change":"2023-06-28T20:07:21.392001+0000","last_active":"2023-06-28T20:07:22.381322+0000","last_peered":"2023-06-28T20:07:22.381322+0000","last_clean":"2023-06-28T20:07:22.381322+0000","last_became_active":"2023-06-28T20:07:21.391521+0000","last_became_peered":"2023-06-28T20:07:21.391521+0000","last_unstale":"2023-06-28T20:07:22.381322+0000","last_undegraded":"2023-06-28T20:07:22.381322+0000","last_fullsized":"2023-06-28T20:07:22.381322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:30:02.751518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.386550+0000","last_change":"2023-06-28T20:07:21.381060+0000","last_active":"2023-06-28T20:07:21.386550+0000","last_peered":"2023-06-28T20:07:21.386550+0000","last_clean":"2023-06-28T20:07:21.386550+0000","last_became_active":"2023-06-28T20:07:21.380791+0000","last_became_peered":"2023-06-28T20:07:21.380791+0000","last_unstale":"2023-06-28T20:07:21.386550+0000","last_undegraded":"2023-06-28T20:07:21.386550+0000","last_fullsized":"2023-06-28T20:07:21.386550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:30:40.906018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.388311+0000","last_change":"2023-06-28T20:07:21.375652+0000","last_active":"2023-06-28T20:07:21.388311+0000","last_peered":"2023-06-28T20:07:21.388311+0000","last_clean":"2023-06-28T20:07:21.388311+0000","last_became_active":"2023-06-28T20:07:21.375192+0000","last_became_peered":"2023-06-28T20:07:21.375192+0000","last_unstale":"2023-06-28T20:07:21.388311+0000","last_undegraded":"2023-06-28T20:07:21.388311+0000","last_fullsized":"2023-06-28T20:07:21.388311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:14:35.290431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382914+0000","last_change":"2023-06-28T20:07:21.376200+0000","last_active":"2023-06-28T20:07:22.382914+0000","last_peered":"2023-06-28T20:07:22.382914+0000","last_clean":"2023-06-28T20:07:22.382914+0000","last_became_active":"2023-06-28T20:07:21.375927+0000","last_became_peered":"2023-06-28T20:07:21.375927+0000","last_unstale":"2023-06-28T20:07:22.382914+0000","last_undegraded":"2023-06-28T20:07:22.382914+0000","last_fullsized":"2023-06-28T20:07:22.382914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:04:16.061920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.395680+0000","last_change":"2023-06-28T20:07:21.394022+0000","last_active":"2023-06-28T20:07:21.395680+0000","last_peered":"2023-06-28T20:07:21.395680+0000","last_clean":"2023-06-28T20:07:21.395680+0000","last_became_active":"2023-06-28T20:07:21.393749+0000","last_became_peered":"2023-06-28T20:07:21.393749+0000","last_unstale":"2023-06-28T20:07:21.395680+0000","last_undegraded":"2023-06-28T20:07:21.395680+0000","last_fullsized":"2023-06-28T20:07:21.395680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:30:02.751518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.388662+0000","last_change":"2023-06-28T20:07:21.380285+0000","last_active":"2023-06-28T20:07:21.388662+0000","last_peered":"2023-06-28T20:07:21.388662+0000","last_clean":"2023-06-28T20:07:21.388662+0000","last_became_active":"2023-06-28T20:07:21.379978+0000","last_became_peered":"2023-06-28T20:07:21.379978+0000","last_unstale":"2023-06-28T20:07:21.388662+0000","last_undegraded":"2023-06-28T20:07:21.388662+0000","last_fullsized":"2023-06-28T20:07:21.388662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:43:18.447196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381458+0000","last_change":"2023-06-28T20:07:21.380983+0000","last_active":"2023-06-28T20:07:22.381458+0000","last_peered":"2023-06-28T20:07:22.381458+0000","last_clean":"2023-06-28T20:07:22.381458+0000","last_became_active":"2023-06-28T20:07:21.380706+0000","last_became_peered":"2023-06-28T20:07:21.380706+0000","last_unstale":"2023-06-28T20:07:22.381458+0000","last_undegraded":"2023-06-28T20:07:22.381458+0000","last_fullsized":"2023-06-28T20:07:22.381458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:43:51.244165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.385477+0000","last_change":"2023-06-28T20:07:21.376404+0000","last_active":"2023-06-28T20:07:21.385477+0000","last_peered":"2023-06-28T20:07:21.385477+0000","last_clean":"2023-06-28T20:07:21.385477+0000","last_became_active":"2023-06-28T20:07:21.376136+0000","last_became_peered":"2023-06-28T20:07:21.376136+0000","last_unstale":"2023-06-28T20:07:21.385477+0000","last_undegraded":"2023-06-28T20:07:21.385477+0000","last_fullsized":"2023-06-28T20:07:21.385477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:05:36.597881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.395486+0000","last_change":"2023-06-28T20:07:21.394452+0000","last_active":"2023-06-28T20:07:21.395486+0000","last_peered":"2023-06-28T20:07:21.395486+0000","last_clean":"2023-06-28T20:07:21.395486+0000","last_became_active":"2023-06-28T20:07:21.393190+0000","last_became_peered":"2023-06-28T20:07:21.393190+0000","last_unstale":"2023-06-28T20:07:21.395486+0000","last_undegraded":"2023-06-28T20:07:21.395486+0000","last_fullsized":"2023-06-28T20:07:21.395486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:05:29.189145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.388674+0000","last_change":"2023-06-28T20:07:21.384870+0000","last_active":"2023-06-28T20:07:21.388674+0000","last_peered":"2023-06-28T20:07:21.388674+0000","last_clean":"2023-06-28T20:07:21.388674+0000","last_became_active":"2023-06-28T20:07:21.384463+0000","last_became_peered":"2023-06-28T20:07:21.384463+0000","last_unstale":"2023-06-28T20:07:21.388674+0000","last_undegraded":"2023-06-28T20:07:21.388674+0000","last_fullsized":"2023-06-28T20:07:21.388674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:56:54.040531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.388753+0000","last_change":"2023-06-28T20:07:21.381193+0000","last_active":"2023-06-28T20:07:21.388753+0000","last_peered":"2023-06-28T20:07:21.388753+0000","last_clean":"2023-06-28T20:07:21.388753+0000","last_became_active":"2023-06-28T20:07:21.380917+0000","last_became_peered":"2023-06-28T20:07:21.380917+0000","last_unstale":"2023-06-28T20:07:21.388753+0000","last_undegraded":"2023-06-28T20:07:21.388753+0000","last_fullsized":"2023-06-28T20:07:21.388753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:34:26.678987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381377+0000","last_change":"2023-06-28T20:07:21.382341+0000","last_active":"2023-06-28T20:07:22.381377+0000","last_peered":"2023-06-28T20:07:22.381377+0000","last_clean":"2023-06-28T20:07:22.381377+0000","last_became_active":"2023-06-28T20:07:21.382032+0000","last_became_peered":"2023-06-28T20:07:21.382032+0000","last_unstale":"2023-06-28T20:07:22.381377+0000","last_undegraded":"2023-06-28T20:07:22.381377+0000","last_fullsized":"2023-06-28T20:07:22.381377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T08:00:53.732415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381426+0000","last_change":"2023-06-28T20:07:21.393006+0000","last_active":"2023-06-28T20:07:22.381426+0000","last_peered":"2023-06-28T20:07:22.381426+0000","last_clean":"2023-06-28T20:07:22.381426+0000","last_became_active":"2023-06-28T20:07:21.392710+0000","last_became_peered":"2023-06-28T20:07:21.392710+0000","last_unstale":"2023-06-28T20:07:22.381426+0000","last_undegraded":"2023-06-28T20:07:22.381426+0000","last_fullsized":"2023-06-28T20:07:22.381426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:01:21.282123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.385631+0000","last_change":"2023-06-28T20:07:21.380173+0000","last_active":"2023-06-28T20:07:21.385631+0000","last_peered":"2023-06-28T20:07:21.385631+0000","last_clean":"2023-06-28T20:07:21.385631+0000","last_became_active":"2023-06-28T20:07:21.379821+0000","last_became_peered":"2023-06-28T20:07:21.379821+0000","last_unstale":"2023-06-28T20:07:21.385631+0000","last_undegraded":"2023-06-28T20:07:21.385631+0000","last_fullsized":"2023-06-28T20:07:21.385631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:30:55.763718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381515+0000","last_change":"2023-06-28T20:07:21.381063+0000","last_active":"2023-06-28T20:07:22.381515+0000","last_peered":"2023-06-28T20:07:22.381515+0000","last_clean":"2023-06-28T20:07:22.381515+0000","last_became_active":"2023-06-28T20:07:21.380785+0000","last_became_peered":"2023-06-28T20:07:21.380785+0000","last_unstale":"2023-06-28T20:07:22.381515+0000","last_undegraded":"2023-06-28T20:07:22.381515+0000","last_fullsized":"2023-06-28T20:07:22.381515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:48:48.825002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.388719+0000","last_change":"2023-06-28T20:07:21.380326+0000","last_active":"2023-06-28T20:07:21.388719+0000","last_peered":"2023-06-28T20:07:21.388719+0000","last_clean":"2023-06-28T20:07:21.388719+0000","last_became_active":"2023-06-28T20:07:21.380043+0000","last_became_peered":"2023-06-28T20:07:21.380043+0000","last_unstale":"2023-06-28T20:07:21.388719+0000","last_undegraded":"2023-06-28T20:07:21.388719+0000","last_fullsized":"2023-06-28T20:07:21.388719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:01:21.282123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.395133+0000","last_change":"2023-06-28T20:07:21.393453+0000","last_active":"2023-06-28T20:07:21.395133+0000","last_peered":"2023-06-28T20:07:21.395133+0000","last_clean":"2023-06-28T20:07:21.395133+0000","last_became_active":"2023-06-28T20:07:21.392856+0000","last_became_peered":"2023-06-28T20:07:21.392856+0000","last_unstale":"2023-06-28T20:07:21.395133+0000","last_undegraded":"2023-06-28T20:07:21.395133+0000","last_fullsized":"2023-06-28T20:07:21.395133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:30:55.763718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.390040+0000","last_change":"2023-06-28T20:07:21.387823+0000","last_active":"2023-06-28T20:07:21.390040+0000","last_peered":"2023-06-28T20:07:21.390040+0000","last_clean":"2023-06-28T20:07:21.390040+0000","last_became_active":"2023-06-28T20:07:21.387528+0000","last_became_peered":"2023-06-28T20:07:21.387528+0000","last_unstale":"2023-06-28T20:07:21.390040+0000","last_undegraded":"2023-06-28T20:07:21.390040+0000","last_fullsized":"2023-06-28T20:07:21.390040+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:16:04.779207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381385+0000","last_change":"2023-06-28T20:07:21.383174+0000","last_active":"2023-06-28T20:07:22.381385+0000","last_peered":"2023-06-28T20:07:22.381385+0000","last_clean":"2023-06-28T20:07:22.381385+0000","last_became_active":"2023-06-28T20:07:21.382865+0000","last_became_peered":"2023-06-28T20:07:21.382865+0000","last_unstale":"2023-06-28T20:07:22.381385+0000","last_undegraded":"2023-06-28T20:07:22.381385+0000","last_fullsized":"2023-06-28T20:07:22.381385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:19:03.691963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.388621+0000","last_change":"2023-06-28T20:07:21.379462+0000","last_active":"2023-06-28T20:07:21.388621+0000","last_peered":"2023-06-28T20:07:21.388621+0000","last_clean":"2023-06-28T20:07:21.388621+0000","last_became_active":"2023-06-28T20:07:21.379190+0000","last_became_peered":"2023-06-28T20:07:21.379190+0000","last_unstale":"2023-06-28T20:07:21.388621+0000","last_undegraded":"2023-06-28T20:07:21.388621+0000","last_fullsized":"2023-06-28T20:07:21.388621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:55:38.748243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381211+0000","last_change":"2023-06-28T20:07:21.392576+0000","last_active":"2023-06-28T20:07:22.381211+0000","last_peered":"2023-06-28T20:07:22.381211+0000","last_clean":"2023-06-28T20:07:22.381211+0000","last_became_active":"2023-06-28T20:07:21.392374+0000","last_became_peered":"2023-06-28T20:07:21.392374+0000","last_unstale":"2023-06-28T20:07:22.381211+0000","last_undegraded":"2023-06-28T20:07:22.381211+0000","last_fullsized":"2023-06-28T20:07:22.381211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:56:31.654337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.387803+0000","last_change":"2023-06-28T20:07:21.381495+0000","last_active":"2023-06-28T20:07:21.387803+0000","last_peered":"2023-06-28T20:07:21.387803+0000","last_clean":"2023-06-28T20:07:21.387803+0000","last_became_active":"2023-06-28T20:07:21.380960+0000","last_became_peered":"2023-06-28T20:07:21.380960+0000","last_unstale":"2023-06-28T20:07:21.387803+0000","last_undegraded":"2023-06-28T20:07:21.387803+0000","last_fullsized":"2023-06-28T20:07:21.387803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T08:00:53.732415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.395494+0000","last_change":"2023-06-28T20:07:21.393139+0000","last_active":"2023-06-28T20:07:21.395494+0000","last_peered":"2023-06-28T20:07:21.395494+0000","last_clean":"2023-06-28T20:07:21.395494+0000","last_became_active":"2023-06-28T20:07:21.392866+0000","last_became_peered":"2023-06-28T20:07:21.392866+0000","last_unstale":"2023-06-28T20:07:21.395494+0000","last_undegraded":"2023-06-28T20:07:21.395494+0000","last_fullsized":"2023-06-28T20:07:21.395494+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:46:12.205864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.380893+0000","last_change":"2023-06-28T20:07:21.386233+0000","last_active":"2023-06-28T20:07:22.380893+0000","last_peered":"2023-06-28T20:07:22.380893+0000","last_clean":"2023-06-28T20:07:22.380893+0000","last_became_active":"2023-06-28T20:07:21.385374+0000","last_became_peered":"2023-06-28T20:07:21.385374+0000","last_unstale":"2023-06-28T20:07:22.380893+0000","last_undegraded":"2023-06-28T20:07:22.380893+0000","last_fullsized":"2023-06-28T20:07:22.380893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:31:11.922720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.397454+0000","last_change":"2023-06-28T20:07:21.381465+0000","last_active":"2023-06-28T20:07:21.397454+0000","last_peered":"2023-06-28T20:07:21.397454+0000","last_clean":"2023-06-28T20:07:21.397454+0000","last_became_active":"2023-06-28T20:07:21.378500+0000","last_became_peered":"2023-06-28T20:07:21.378500+0000","last_unstale":"2023-06-28T20:07:21.397454+0000","last_undegraded":"2023-06-28T20:07:21.397454+0000","last_fullsized":"2023-06-28T20:07:21.397454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:05:36.597881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.397634+0000","last_change":"2023-06-28T20:07:21.384064+0000","last_active":"2023-06-28T20:07:21.397634+0000","last_peered":"2023-06-28T20:07:21.397634+0000","last_clean":"2023-06-28T20:07:21.397634+0000","last_became_active":"2023-06-28T20:07:21.383797+0000","last_became_peered":"2023-06-28T20:07:21.383797+0000","last_unstale":"2023-06-28T20:07:21.397634+0000","last_undegraded":"2023-06-28T20:07:21.397634+0000","last_fullsized":"2023-06-28T20:07:21.397634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:56:54.040531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.388470+0000","last_change":"2023-06-28T20:07:21.382031+0000","last_active":"2023-06-28T20:07:21.388470+0000","last_peered":"2023-06-28T20:07:21.388470+0000","last_clean":"2023-06-28T20:07:21.388470+0000","last_became_active":"2023-06-28T20:07:21.381763+0000","last_became_peered":"2023-06-28T20:07:21.381763+0000","last_unstale":"2023-06-28T20:07:21.388470+0000","last_undegraded":"2023-06-28T20:07:21.388470+0000","last_fullsized":"2023-06-28T20:07:21.388470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:39:10.624775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.397890+0000","last_change":"2023-06-28T20:07:20.382468+0000","last_active":"2023-06-28T20:07:21.397890+0000","last_peered":"2023-06-28T20:07:21.397890+0000","last_clean":"2023-06-28T20:07:21.397890+0000","last_became_active":"2023-06-28T20:07:20.382185+0000","last_became_peered":"2023-06-28T20:07:20.382185+0000","last_unstale":"2023-06-28T20:07:21.397890+0000","last_undegraded":"2023-06-28T20:07:21.397890+0000","last_fullsized":"2023-06-28T20:07:21.397890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:05:45.610772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.392080+0000","last_change":"2023-06-28T20:07:21.386960+0000","last_active":"2023-06-28T20:07:21.392080+0000","last_peered":"2023-06-28T20:07:21.392080+0000","last_clean":"2023-06-28T20:07:21.392080+0000","last_became_active":"2023-06-28T20:07:21.386705+0000","last_became_peered":"2023-06-28T20:07:21.386705+0000","last_unstale":"2023-06-28T20:07:21.392080+0000","last_undegraded":"2023-06-28T20:07:21.392080+0000","last_fullsized":"2023-06-28T20:07:21.392080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:30:55.763718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.364204+0000","last_change":"2023-06-28T20:07:20.382275+0000","last_active":"2023-06-28T20:07:21.364204+0000","last_peered":"2023-06-28T20:07:21.364204+0000","last_clean":"2023-06-28T20:07:21.364204+0000","last_became_active":"2023-06-28T20:07:20.381923+0000","last_became_peered":"2023-06-28T20:07:20.381923+0000","last_unstale":"2023-06-28T20:07:21.364204+0000","last_undegraded":"2023-06-28T20:07:21.364204+0000","last_fullsized":"2023-06-28T20:07:21.364204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T08:00:52.726933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.383166+0000","last_change":"2023-06-28T20:07:21.380199+0000","last_active":"2023-06-28T20:07:21.383166+0000","last_peered":"2023-06-28T20:07:21.383166+0000","last_clean":"2023-06-28T20:07:21.383166+0000","last_became_active":"2023-06-28T20:07:21.379925+0000","last_became_peered":"2023-06-28T20:07:21.379925+0000","last_unstale":"2023-06-28T20:07:21.383166+0000","last_undegraded":"2023-06-28T20:07:21.383166+0000","last_fullsized":"2023-06-28T20:07:21.383166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:01:21.282123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.368710+0000","last_change":"2023-06-28T20:07:20.382465+0000","last_active":"2023-06-28T20:07:21.368710+0000","last_peered":"2023-06-28T20:07:21.368710+0000","last_clean":"2023-06-28T20:07:21.368710+0000","last_became_active":"2023-06-28T20:07:20.380273+0000","last_became_peered":"2023-06-28T20:07:20.380273+0000","last_unstale":"2023-06-28T20:07:21.368710+0000","last_undegraded":"2023-06-28T20:07:21.368710+0000","last_fullsized":"2023-06-28T20:07:21.368710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:56:53.035050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382428+0000","last_change":"2023-06-28T20:07:21.386881+0000","last_active":"2023-06-28T20:07:22.382428+0000","last_peered":"2023-06-28T20:07:22.382428+0000","last_clean":"2023-06-28T20:07:22.382428+0000","last_became_active":"2023-06-28T20:07:21.383395+0000","last_became_peered":"2023-06-28T20:07:21.383395+0000","last_unstale":"2023-06-28T20:07:22.382428+0000","last_undegraded":"2023-06-28T20:07:22.382428+0000","last_fullsized":"2023-06-28T20:07:22.382428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:34:26.678987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.377756+0000","last_change":"2023-06-28T20:07:20.378567+0000","last_active":"2023-06-28T20:07:21.377756+0000","last_peered":"2023-06-28T20:07:21.377756+0000","last_clean":"2023-06-28T20:07:21.377756+0000","last_became_active":"2023-06-28T20:07:20.377624+0000","last_became_peered":"2023-06-28T20:07:20.377624+0000","last_unstale":"2023-06-28T20:07:21.377756+0000","last_undegraded":"2023-06-28T20:07:21.377756+0000","last_fullsized":"2023-06-28T20:07:21.377756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:56:53.035050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.391370+0000","last_change":"2023-06-28T20:07:21.385250+0000","last_active":"2023-06-28T20:07:21.391370+0000","last_peered":"2023-06-28T20:07:21.391370+0000","last_clean":"2023-06-28T20:07:21.391370+0000","last_became_active":"2023-06-28T20:07:21.384763+0000","last_became_peered":"2023-06-28T20:07:21.384763+0000","last_unstale":"2023-06-28T20:07:21.391370+0000","last_undegraded":"2023-06-28T20:07:21.391370+0000","last_fullsized":"2023-06-28T20:07:21.391370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:05:36.597881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.371341+0000","last_change":"2023-06-28T20:07:20.376535+0000","last_active":"2023-06-28T20:07:21.371341+0000","last_peered":"2023-06-28T20:07:21.371341+0000","last_clean":"2023-06-28T20:07:21.371341+0000","last_became_active":"2023-06-28T20:07:20.376182+0000","last_became_peered":"2023-06-28T20:07:20.376182+0000","last_unstale":"2023-06-28T20:07:21.371341+0000","last_undegraded":"2023-06-28T20:07:21.371341+0000","last_fullsized":"2023-06-28T20:07:21.371341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:43:17.441714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.394657+0000","last_change":"2023-06-28T20:07:21.390529+0000","last_active":"2023-06-28T20:07:21.394657+0000","last_peered":"2023-06-28T20:07:21.394657+0000","last_clean":"2023-06-28T20:07:21.394657+0000","last_became_active":"2023-06-28T20:07:21.390249+0000","last_became_peered":"2023-06-28T20:07:21.390249+0000","last_unstale":"2023-06-28T20:07:21.394657+0000","last_undegraded":"2023-06-28T20:07:21.394657+0000","last_fullsized":"2023-06-28T20:07:21.394657+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:01:21.282123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381357+0000","last_change":"2023-06-28T20:07:20.380921+0000","last_active":"2023-06-28T20:07:22.381357+0000","last_peered":"2023-06-28T20:07:22.381357+0000","last_clean":"2023-06-28T20:07:22.381357+0000","last_became_active":"2023-06-28T20:07:20.380632+0000","last_became_peered":"2023-06-28T20:07:20.380632+0000","last_unstale":"2023-06-28T20:07:22.381357+0000","last_undegraded":"2023-06-28T20:07:22.381357+0000","last_fullsized":"2023-06-28T20:07:22.381357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:03:07.390037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.396451+0000","last_change":"2023-06-28T20:07:21.379317+0000","last_active":"2023-06-28T20:07:21.396451+0000","last_peered":"2023-06-28T20:07:21.396451+0000","last_clean":"2023-06-28T20:07:21.396451+0000","last_became_active":"2023-06-28T20:07:21.379040+0000","last_became_peered":"2023-06-28T20:07:21.379040+0000","last_unstale":"2023-06-28T20:07:21.396451+0000","last_undegraded":"2023-06-28T20:07:21.396451+0000","last_fullsized":"2023-06-28T20:07:21.396451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:48:03.203326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.396786+0000","last_change":"2023-06-28T20:07:20.376485+0000","last_active":"2023-06-28T20:07:21.396786+0000","last_peered":"2023-06-28T20:07:21.396786+0000","last_clean":"2023-06-28T20:07:21.396786+0000","last_became_active":"2023-06-28T20:07:20.376240+0000","last_became_peered":"2023-06-28T20:07:20.376240+0000","last_unstale":"2023-06-28T20:07:21.396786+0000","last_undegraded":"2023-06-28T20:07:21.396786+0000","last_fullsized":"2023-06-28T20:07:21.396786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:40:58.608291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.388583+0000","last_change":"2023-06-28T20:07:21.379150+0000","last_active":"2023-06-28T20:07:21.388583+0000","last_peered":"2023-06-28T20:07:21.388583+0000","last_clean":"2023-06-28T20:07:21.388583+0000","last_became_active":"2023-06-28T20:07:21.378850+0000","last_became_peered":"2023-06-28T20:07:21.378850+0000","last_unstale":"2023-06-28T20:07:21.388583+0000","last_undegraded":"2023-06-28T20:07:21.388583+0000","last_fullsized":"2023-06-28T20:07:21.388583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:48:03.203326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381032+0000","last_change":"2023-06-28T20:07:20.378538+0000","last_active":"2023-06-28T20:07:22.381032+0000","last_peered":"2023-06-28T20:07:22.381032+0000","last_clean":"2023-06-28T20:07:22.381032+0000","last_became_active":"2023-06-28T20:07:20.378180+0000","last_became_peered":"2023-06-28T20:07:20.378180+0000","last_unstale":"2023-06-28T20:07:22.381032+0000","last_undegraded":"2023-06-28T20:07:22.381032+0000","last_fullsized":"2023-06-28T20:07:22.381032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:58:49.563930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.393234+0000","last_change":"2023-06-28T20:07:21.389643+0000","last_active":"2023-06-28T20:07:21.393234+0000","last_peered":"2023-06-28T20:07:21.393234+0000","last_clean":"2023-06-28T20:07:21.393234+0000","last_became_active":"2023-06-28T20:07:21.389163+0000","last_became_peered":"2023-06-28T20:07:21.389163+0000","last_unstale":"2023-06-28T20:07:21.393234+0000","last_undegraded":"2023-06-28T20:07:21.393234+0000","last_fullsized":"2023-06-28T20:07:21.393234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:30:40.906018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.377704+0000","last_change":"2023-06-28T20:07:20.371972+0000","last_active":"2023-06-28T20:07:21.377704+0000","last_peered":"2023-06-28T20:07:21.377704+0000","last_clean":"2023-06-28T20:07:21.377704+0000","last_became_active":"2023-06-28T20:07:20.370619+0000","last_became_peered":"2023-06-28T20:07:20.370619+0000","last_unstale":"2023-06-28T20:07:21.377704+0000","last_undegraded":"2023-06-28T20:07:21.377704+0000","last_fullsized":"2023-06-28T20:07:21.377704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:30:39.900537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.388496+0000","last_change":"2023-06-28T20:07:21.382142+0000","last_active":"2023-06-28T20:07:21.388496+0000","last_peered":"2023-06-28T20:07:21.388496+0000","last_clean":"2023-06-28T20:07:21.388496+0000","last_became_active":"2023-06-28T20:07:21.381848+0000","last_became_peered":"2023-06-28T20:07:21.381848+0000","last_unstale":"2023-06-28T20:07:21.388496+0000","last_undegraded":"2023-06-28T20:07:21.388496+0000","last_fullsized":"2023-06-28T20:07:21.388496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:05:29.189145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381309+0000","last_change":"2023-06-28T20:07:20.376749+0000","last_active":"2023-06-28T20:07:22.381309+0000","last_peered":"2023-06-28T20:07:22.381309+0000","last_clean":"2023-06-28T20:07:22.381309+0000","last_became_active":"2023-06-28T20:07:20.376465+0000","last_became_peered":"2023-06-28T20:07:20.376465+0000","last_unstale":"2023-06-28T20:07:22.381309+0000","last_undegraded":"2023-06-28T20:07:22.381309+0000","last_fullsized":"2023-06-28T20:07:22.381309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:20:39.309655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.395123+0000","last_change":"2023-06-28T20:07:21.393127+0000","last_active":"2023-06-28T20:07:21.395123+0000","last_peered":"2023-06-28T20:07:21.395123+0000","last_clean":"2023-06-28T20:07:21.395123+0000","last_became_active":"2023-06-28T20:07:21.392832+0000","last_became_peered":"2023-06-28T20:07:21.392832+0000","last_unstale":"2023-06-28T20:07:21.395123+0000","last_undegraded":"2023-06-28T20:07:21.395123+0000","last_fullsized":"2023-06-28T20:07:21.395123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:05:36.597881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381838+0000","last_change":"2023-06-28T20:07:20.380053+0000","last_active":"2023-06-28T20:07:22.381838+0000","last_peered":"2023-06-28T20:07:22.381838+0000","last_clean":"2023-06-28T20:07:22.381838+0000","last_became_active":"2023-06-28T20:07:20.379752+0000","last_became_peered":"2023-06-28T20:07:20.379752+0000","last_unstale":"2023-06-28T20:07:22.381838+0000","last_undegraded":"2023-06-28T20:07:22.381838+0000","last_fullsized":"2023-06-28T20:07:22.381838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:58:14.638684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.395185+0000","last_change":"2023-06-28T20:07:21.390679+0000","last_active":"2023-06-28T20:07:21.395185+0000","last_peered":"2023-06-28T20:07:21.395185+0000","last_clean":"2023-06-28T20:07:21.395185+0000","last_became_active":"2023-06-28T20:07:21.390332+0000","last_became_peered":"2023-06-28T20:07:21.390332+0000","last_unstale":"2023-06-28T20:07:21.395185+0000","last_undegraded":"2023-06-28T20:07:21.395185+0000","last_fullsized":"2023-06-28T20:07:21.395185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:58:15.644165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381372+0000","last_change":"2023-06-28T20:07:20.379001+0000","last_active":"2023-06-28T20:07:22.381372+0000","last_peered":"2023-06-28T20:07:22.381372+0000","last_clean":"2023-06-28T20:07:22.381372+0000","last_became_active":"2023-06-28T20:07:20.378692+0000","last_became_peered":"2023-06-28T20:07:20.378692+0000","last_unstale":"2023-06-28T20:07:22.381372+0000","last_undegraded":"2023-06-28T20:07:22.381372+0000","last_fullsized":"2023-06-28T20:07:22.381372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:50:31.576319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.385500+0000","last_change":"2023-06-28T20:07:21.377599+0000","last_active":"2023-06-28T20:07:21.385500+0000","last_peered":"2023-06-28T20:07:21.385500+0000","last_clean":"2023-06-28T20:07:21.385500+0000","last_became_active":"2023-06-28T20:07:21.377323+0000","last_became_peered":"2023-06-28T20:07:21.377323+0000","last_unstale":"2023-06-28T20:07:21.385500+0000","last_undegraded":"2023-06-28T20:07:21.385500+0000","last_fullsized":"2023-06-28T20:07:21.385500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:01:21.282123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381358+0000","last_change":"2023-06-28T20:07:20.379564+0000","last_active":"2023-06-28T20:07:22.381358+0000","last_peered":"2023-06-28T20:07:22.381358+0000","last_clean":"2023-06-28T20:07:22.381358+0000","last_became_active":"2023-06-28T20:07:20.379179+0000","last_became_peered":"2023-06-28T20:07:20.379179+0000","last_unstale":"2023-06-28T20:07:22.381358+0000","last_undegraded":"2023-06-28T20:07:22.381358+0000","last_fullsized":"2023-06-28T20:07:22.381358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:44:14.718714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.387776+0000","last_change":"2023-06-28T20:07:21.375868+0000","last_active":"2023-06-28T20:07:21.387776+0000","last_peered":"2023-06-28T20:07:21.387776+0000","last_clean":"2023-06-28T20:07:21.387776+0000","last_became_active":"2023-06-28T20:07:21.375273+0000","last_became_peered":"2023-06-28T20:07:21.375273+0000","last_unstale":"2023-06-28T20:07:21.387776+0000","last_undegraded":"2023-06-28T20:07:21.387776+0000","last_fullsized":"2023-06-28T20:07:21.387776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:05:29.189145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381183+0000","last_change":"2023-06-28T20:07:20.377080+0000","last_active":"2023-06-28T20:07:22.381183+0000","last_peered":"2023-06-28T20:07:22.381183+0000","last_clean":"2023-06-28T20:07:22.381183+0000","last_became_active":"2023-06-28T20:07:20.376266+0000","last_became_peered":"2023-06-28T20:07:20.376266+0000","last_unstale":"2023-06-28T20:07:22.381183+0000","last_undegraded":"2023-06-28T20:07:22.381183+0000","last_fullsized":"2023-06-28T20:07:22.381183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:21:31.960769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382189+0000","last_change":"2023-06-28T20:07:21.382463+0000","last_active":"2023-06-28T20:07:22.382189+0000","last_peered":"2023-06-28T20:07:22.382189+0000","last_clean":"2023-06-28T20:07:22.382189+0000","last_became_active":"2023-06-28T20:07:21.382163+0000","last_became_peered":"2023-06-28T20:07:21.382163+0000","last_unstale":"2023-06-28T20:07:22.382189+0000","last_undegraded":"2023-06-28T20:07:22.382189+0000","last_fullsized":"2023-06-28T20:07:22.382189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:44:41.666645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.369052+0000","last_change":"2023-06-28T20:07:20.380798+0000","last_active":"2023-06-28T20:07:21.369052+0000","last_peered":"2023-06-28T20:07:21.369052+0000","last_clean":"2023-06-28T20:07:21.369052+0000","last_became_active":"2023-06-28T20:07:20.380380+0000","last_became_peered":"2023-06-28T20:07:20.380380+0000","last_unstale":"2023-06-28T20:07:21.369052+0000","last_undegraded":"2023-06-28T20:07:21.369052+0000","last_fullsized":"2023-06-28T20:07:21.369052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:48:02.197845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.392246+0000","last_change":"2023-06-28T20:07:21.382097+0000","last_active":"2023-06-28T20:07:21.392246+0000","last_peered":"2023-06-28T20:07:21.392246+0000","last_clean":"2023-06-28T20:07:21.392246+0000","last_became_active":"2023-06-28T20:07:21.381678+0000","last_became_peered":"2023-06-28T20:07:21.381678+0000","last_unstale":"2023-06-28T20:07:21.392246+0000","last_undegraded":"2023-06-28T20:07:21.392246+0000","last_fullsized":"2023-06-28T20:07:21.392246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:19:03.691963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.397699+0000","last_change":"2023-06-28T20:07:20.380519+0000","last_active":"2023-06-28T20:07:21.397699+0000","last_peered":"2023-06-28T20:07:21.397699+0000","last_clean":"2023-06-28T20:07:21.397699+0000","last_became_active":"2023-06-28T20:07:20.380219+0000","last_became_peered":"2023-06-28T20:07:20.380219+0000","last_unstale":"2023-06-28T20:07:21.397699+0000","last_undegraded":"2023-06-28T20:07:21.397699+0000","last_fullsized":"2023-06-28T20:07:21.397699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:44:40.661163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.392115+0000","last_change":"2023-06-28T20:07:21.391559+0000","last_active":"2023-06-28T20:07:21.392115+0000","last_peered":"2023-06-28T20:07:21.392115+0000","last_clean":"2023-06-28T20:07:21.392115+0000","last_became_active":"2023-06-28T20:07:21.391214+0000","last_became_peered":"2023-06-28T20:07:21.391214+0000","last_unstale":"2023-06-28T20:07:21.392115+0000","last_undegraded":"2023-06-28T20:07:21.392115+0000","last_fullsized":"2023-06-28T20:07:21.392115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:07:49.232842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382144+0000","last_change":"2023-06-28T20:07:20.378412+0000","last_active":"2023-06-28T20:07:22.382144+0000","last_peered":"2023-06-28T20:07:22.382144+0000","last_clean":"2023-06-28T20:07:22.382144+0000","last_became_active":"2023-06-28T20:07:20.378102+0000","last_became_peered":"2023-06-28T20:07:20.378102+0000","last_unstale":"2023-06-28T20:07:22.382144+0000","last_undegraded":"2023-06-28T20:07:22.382144+0000","last_fullsized":"2023-06-28T20:07:22.382144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:05:45.610772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.390394+0000","last_change":"2023-06-28T20:07:21.387960+0000","last_active":"2023-06-28T20:07:21.390394+0000","last_peered":"2023-06-28T20:07:21.390394+0000","last_clean":"2023-06-28T20:07:21.390394+0000","last_became_active":"2023-06-28T20:07:21.387590+0000","last_became_peered":"2023-06-28T20:07:21.387590+0000","last_unstale":"2023-06-28T20:07:21.390394+0000","last_undegraded":"2023-06-28T20:07:21.390394+0000","last_fullsized":"2023-06-28T20:07:21.390394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:46:12.205864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.366745+0000","last_change":"2023-06-28T20:07:20.378036+0000","last_active":"2023-06-28T20:07:21.366745+0000","last_peered":"2023-06-28T20:07:21.366745+0000","last_clean":"2023-06-28T20:07:21.366745+0000","last_became_active":"2023-06-28T20:07:20.377739+0000","last_became_peered":"2023-06-28T20:07:20.377739+0000","last_unstale":"2023-06-28T20:07:21.366745+0000","last_undegraded":"2023-06-28T20:07:21.366745+0000","last_fullsized":"2023-06-28T20:07:21.366745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:56:53.035050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.388764+0000","last_change":"2023-06-28T20:07:21.379387+0000","last_active":"2023-06-28T20:07:21.388764+0000","last_peered":"2023-06-28T20:07:21.388764+0000","last_clean":"2023-06-28T20:07:21.388764+0000","last_became_active":"2023-06-28T20:07:21.379090+0000","last_became_peered":"2023-06-28T20:07:21.379090+0000","last_unstale":"2023-06-28T20:07:21.388764+0000","last_undegraded":"2023-06-28T20:07:21.388764+0000","last_fullsized":"2023-06-28T20:07:21.388764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:44:41.666645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.368360+0000","last_change":"2023-06-28T20:07:20.382672+0000","last_active":"2023-06-28T20:07:21.368360+0000","last_peered":"2023-06-28T20:07:21.368360+0000","last_clean":"2023-06-28T20:07:21.368360+0000","last_became_active":"2023-06-28T20:07:20.381085+0000","last_became_peered":"2023-06-28T20:07:20.381085+0000","last_unstale":"2023-06-28T20:07:21.368360+0000","last_undegraded":"2023-06-28T20:07:21.368360+0000","last_fullsized":"2023-06-28T20:07:21.368360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:05:28.183664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.397860+0000","last_change":"2023-06-28T20:07:21.386618+0000","last_active":"2023-06-28T20:07:21.397860+0000","last_peered":"2023-06-28T20:07:21.397860+0000","last_clean":"2023-06-28T20:07:21.397860+0000","last_became_active":"2023-06-28T20:07:21.386002+0000","last_became_peered":"2023-06-28T20:07:21.386002+0000","last_unstale":"2023-06-28T20:07:21.397860+0000","last_undegraded":"2023-06-28T20:07:21.397860+0000","last_fullsized":"2023-06-28T20:07:21.397860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:06:53.375903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.395132+0000","last_change":"2023-06-28T20:07:20.379096+0000","last_active":"2023-06-28T20:07:21.395132+0000","last_peered":"2023-06-28T20:07:21.395132+0000","last_clean":"2023-06-28T20:07:21.395132+0000","last_became_active":"2023-06-28T20:07:20.378681+0000","last_became_peered":"2023-06-28T20:07:20.378681+0000","last_unstale":"2023-06-28T20:07:21.395132+0000","last_undegraded":"2023-06-28T20:07:21.395132+0000","last_fullsized":"2023-06-28T20:07:21.395132+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:06:52.370421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.369688+0000","last_change":"2023-06-28T20:07:15.815425+0000","last_active":"2023-06-28T20:07:21.369688+0000","last_peered":"2023-06-28T20:07:21.369688+0000","last_clean":"2023-06-28T20:07:21.369688+0000","last_became_active":"2023-06-28T20:07:15.815163+0000","last_became_peered":"2023-06-28T20:07:15.815163+0000","last_unstale":"2023-06-28T20:07:21.369688+0000","last_undegraded":"2023-06-28T20:07:21.369688+0000","last_fullsized":"2023-06-28T20:07:21.369688+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:04:09.994295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.391347+0000","last_change":"2023-06-28T20:07:21.385061+0000","last_active":"2023-06-28T20:07:21.391347+0000","last_peered":"2023-06-28T20:07:21.391347+0000","last_clean":"2023-06-28T20:07:21.391347+0000","last_became_active":"2023-06-28T20:07:21.384404+0000","last_became_peered":"2023-06-28T20:07:21.384404+0000","last_unstale":"2023-06-28T20:07:21.391347+0000","last_undegraded":"2023-06-28T20:07:21.391347+0000","last_fullsized":"2023-06-28T20:07:21.391347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:48:55.783926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.374509+0000","last_change":"2023-06-28T20:07:15.303675+0000","last_active":"2023-06-28T20:07:21.374509+0000","last_peered":"2023-06-28T20:07:21.374509+0000","last_clean":"2023-06-28T20:07:21.374509+0000","last_became_active":"2023-06-28T20:07:15.303350+0000","last_became_peered":"2023-06-28T20:07:15.303350+0000","last_unstale":"2023-06-28T20:07:21.374509+0000","last_undegraded":"2023-06-28T20:07:21.374509+0000","last_fullsized":"2023-06-28T20:07:21.374509+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:34:20.611362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.378021+0000","last_change":"2023-06-28T20:07:20.381457+0000","last_active":"2023-06-28T20:07:21.378021+0000","last_peered":"2023-06-28T20:07:21.378021+0000","last_clean":"2023-06-28T20:07:21.378021+0000","last_became_active":"2023-06-28T20:07:20.381235+0000","last_became_peered":"2023-06-28T20:07:20.381235+0000","last_unstale":"2023-06-28T20:07:21.378021+0000","last_undegraded":"2023-06-28T20:07:21.378021+0000","last_fullsized":"2023-06-28T20:07:21.378021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:39:09.619294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.388793+0000","last_change":"2023-06-28T20:07:21.383647+0000","last_active":"2023-06-28T20:07:21.388793+0000","last_peered":"2023-06-28T20:07:21.388793+0000","last_clean":"2023-06-28T20:07:21.388793+0000","last_became_active":"2023-06-28T20:07:21.383392+0000","last_became_peered":"2023-06-28T20:07:21.383392+0000","last_unstale":"2023-06-28T20:07:21.388793+0000","last_undegraded":"2023-06-28T20:07:21.388793+0000","last_fullsized":"2023-06-28T20:07:21.388793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:05:36.597881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.364257+0000","last_change":"2023-06-28T20:07:15.303565+0000","last_active":"2023-06-28T20:07:21.364257+0000","last_peered":"2023-06-28T20:07:21.364257+0000","last_clean":"2023-06-28T20:07:21.364257+0000","last_became_active":"2023-06-28T20:07:15.303200+0000","last_became_peered":"2023-06-28T20:07:15.303200+0000","last_unstale":"2023-06-28T20:07:21.364257+0000","last_undegraded":"2023-06-28T20:07:21.364257+0000","last_fullsized":"2023-06-28T20:07:21.364257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:01:15.214498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":169,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381052+0000","last_change":"2023-06-28T20:07:15.304921+0000","last_active":"2023-06-28T20:07:22.381052+0000","last_peered":"2023-06-28T20:07:22.381052+0000","last_clean":"2023-06-28T20:07:22.381052+0000","last_became_active":"2023-06-28T20:07:15.303959+0000","last_became_peered":"2023-06-28T20:07:15.303959+0000","last_unstale":"2023-06-28T20:07:22.381052+0000","last_undegraded":"2023-06-28T20:07:22.381052+0000","last_fullsized":"2023-06-28T20:07:22.381052+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:44:42.249408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.375146+0000","last_change":"2023-06-28T20:07:20.379375+0000","last_active":"2023-06-28T20:07:21.375146+0000","last_peered":"2023-06-28T20:07:21.375146+0000","last_clean":"2023-06-28T20:07:21.375146+0000","last_became_active":"2023-06-28T20:07:20.378992+0000","last_became_peered":"2023-06-28T20:07:20.378992+0000","last_unstale":"2023-06-28T20:07:21.375146+0000","last_undegraded":"2023-06-28T20:07:21.375146+0000","last_fullsized":"2023-06-28T20:07:21.375146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:48:54.778445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.393013+0000","last_change":"2023-06-28T20:07:21.383270+0000","last_active":"2023-06-28T20:07:21.393013+0000","last_peered":"2023-06-28T20:07:21.393013+0000","last_clean":"2023-06-28T20:07:21.393013+0000","last_became_active":"2023-06-28T20:07:21.382983+0000","last_became_peered":"2023-06-28T20:07:21.382983+0000","last_unstale":"2023-06-28T20:07:21.393013+0000","last_undegraded":"2023-06-28T20:07:21.393013+0000","last_fullsized":"2023-06-28T20:07:21.393013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:48:03.203326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.380861+0000","last_change":"2023-06-28T20:07:15.899048+0000","last_active":"2023-06-28T20:07:22.380861+0000","last_peered":"2023-06-28T20:07:22.380861+0000","last_clean":"2023-06-28T20:07:22.380861+0000","last_became_active":"2023-06-28T20:07:15.898766+0000","last_became_peered":"2023-06-28T20:07:15.898766+0000","last_unstale":"2023-06-28T20:07:22.380861+0000","last_undegraded":"2023-06-28T20:07:22.380861+0000","last_fullsized":"2023-06-28T20:07:22.380861+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:27:00.869776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.364255+0000","last_change":"2023-06-28T20:07:20.369864+0000","last_active":"2023-06-28T20:07:21.364255+0000","last_peered":"2023-06-28T20:07:21.364255+0000","last_clean":"2023-06-28T20:07:21.364255+0000","last_became_active":"2023-06-28T20:07:20.369542+0000","last_became_peered":"2023-06-28T20:07:20.369542+0000","last_unstale":"2023-06-28T20:07:21.364255+0000","last_undegraded":"2023-06-28T20:07:21.364255+0000","last_fullsized":"2023-06-28T20:07:21.364255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:48:54.778445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.397567+0000","last_change":"2023-06-28T20:07:21.383019+0000","last_active":"2023-06-28T20:07:21.397567+0000","last_peered":"2023-06-28T20:07:21.397567+0000","last_clean":"2023-06-28T20:07:21.397567+0000","last_became_active":"2023-06-28T20:07:21.382756+0000","last_became_peered":"2023-06-28T20:07:21.382756+0000","last_unstale":"2023-06-28T20:07:21.397567+0000","last_undegraded":"2023-06-28T20:07:21.397567+0000","last_fullsized":"2023-06-28T20:07:21.397567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:05:29.189145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.380789+0000","last_change":"2023-06-28T20:07:15.899229+0000","last_active":"2023-06-28T20:07:22.380789+0000","last_peered":"2023-06-28T20:07:22.380789+0000","last_clean":"2023-06-28T20:07:22.380789+0000","last_became_active":"2023-06-28T20:07:15.898708+0000","last_became_peered":"2023-06-28T20:07:15.898708+0000","last_unstale":"2023-06-28T20:07:22.380789+0000","last_undegraded":"2023-06-28T20:07:22.380789+0000","last_fullsized":"2023-06-28T20:07:22.380789+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:28:19.249906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.369286+0000","last_change":"2023-06-28T20:07:20.381183+0000","last_active":"2023-06-28T20:07:21.369286+0000","last_peered":"2023-06-28T20:07:21.369286+0000","last_clean":"2023-06-28T20:07:21.369286+0000","last_became_active":"2023-06-28T20:07:20.380859+0000","last_became_peered":"2023-06-28T20:07:20.380859+0000","last_unstale":"2023-06-28T20:07:21.369286+0000","last_undegraded":"2023-06-28T20:07:21.369286+0000","last_fullsized":"2023-06-28T20:07:21.369286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:39:09.619294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382351+0000","last_change":"2023-06-28T20:07:21.382900+0000","last_active":"2023-06-28T20:07:22.382351+0000","last_peered":"2023-06-28T20:07:22.382351+0000","last_clean":"2023-06-28T20:07:22.382351+0000","last_became_active":"2023-06-28T20:07:21.382384+0000","last_became_peered":"2023-06-28T20:07:21.382384+0000","last_unstale":"2023-06-28T20:07:22.382351+0000","last_undegraded":"2023-06-28T20:07:22.382351+0000","last_fullsized":"2023-06-28T20:07:22.382351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:16:04.779207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.368676+0000","last_change":"2023-06-28T20:07:15.898966+0000","last_active":"2023-06-28T20:07:21.368676+0000","last_peered":"2023-06-28T20:07:21.368676+0000","last_clean":"2023-06-28T20:07:21.368676+0000","last_became_active":"2023-06-28T20:07:15.898704+0000","last_became_peered":"2023-06-28T20:07:15.898704+0000","last_unstale":"2023-06-28T20:07:21.368676+0000","last_undegraded":"2023-06-28T20:07:21.368676+0000","last_fullsized":"2023-06-28T20:07:21.368676+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T08:00:47.664790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.367218+0000","last_change":"2023-06-28T20:07:20.378848+0000","last_active":"2023-06-28T20:07:21.367218+0000","last_peered":"2023-06-28T20:07:21.367218+0000","last_clean":"2023-06-28T20:07:21.367218+0000","last_became_active":"2023-06-28T20:07:20.378438+0000","last_became_peered":"2023-06-28T20:07:20.378438+0000","last_unstale":"2023-06-28T20:07:21.367218+0000","last_undegraded":"2023-06-28T20:07:21.367218+0000","last_fullsized":"2023-06-28T20:07:21.367218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:58:14.638684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.397694+0000","last_change":"2023-06-28T20:07:21.384171+0000","last_active":"2023-06-28T20:07:21.397694+0000","last_peered":"2023-06-28T20:07:21.397694+0000","last_clean":"2023-06-28T20:07:21.397694+0000","last_became_active":"2023-06-28T20:07:21.383867+0000","last_became_peered":"2023-06-28T20:07:21.383867+0000","last_unstale":"2023-06-28T20:07:21.397694+0000","last_undegraded":"2023-06-28T20:07:21.397694+0000","last_fullsized":"2023-06-28T20:07:21.397694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:39:10.624775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382412+0000","last_change":"2023-06-28T20:07:15.899207+0000","last_active":"2023-06-28T20:07:22.382412+0000","last_peered":"2023-06-28T20:07:22.382412+0000","last_clean":"2023-06-28T20:07:22.382412+0000","last_became_active":"2023-06-28T20:07:15.898908+0000","last_became_peered":"2023-06-28T20:07:15.898908+0000","last_unstale":"2023-06-28T20:07:22.382412+0000","last_undegraded":"2023-06-28T20:07:22.382412+0000","last_fullsized":"2023-06-28T20:07:22.382412+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:14:29.222806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.365573+0000","last_change":"2023-06-28T20:07:20.377530+0000","last_active":"2023-06-28T20:07:21.365573+0000","last_peered":"2023-06-28T20:07:21.365573+0000","last_clean":"2023-06-28T20:07:21.365573+0000","last_became_active":"2023-06-28T20:07:20.377243+0000","last_became_peered":"2023-06-28T20:07:20.377243+0000","last_unstale":"2023-06-28T20:07:21.365573+0000","last_undegraded":"2023-06-28T20:07:21.365573+0000","last_fullsized":"2023-06-28T20:07:21.365573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:58:14.638684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.392290+0000","last_change":"2023-06-28T20:07:21.383886+0000","last_active":"2023-06-28T20:07:21.392290+0000","last_peered":"2023-06-28T20:07:21.392290+0000","last_clean":"2023-06-28T20:07:21.392290+0000","last_became_active":"2023-06-28T20:07:21.383572+0000","last_became_peered":"2023-06-28T20:07:21.383572+0000","last_unstale":"2023-06-28T20:07:21.392290+0000","last_undegraded":"2023-06-28T20:07:21.392290+0000","last_fullsized":"2023-06-28T20:07:21.392290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:06:01.974529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382342+0000","last_change":"2023-06-28T20:07:15.303873+0000","last_active":"2023-06-28T20:07:22.382342+0000","last_peered":"2023-06-28T20:07:22.382342+0000","last_clean":"2023-06-28T20:07:22.382342+0000","last_became_active":"2023-06-28T20:07:15.303457+0000","last_became_peered":"2023-06-28T20:07:15.303457+0000","last_unstale":"2023-06-28T20:07:22.382342+0000","last_undegraded":"2023-06-28T20:07:22.382342+0000","last_fullsized":"2023-06-28T20:07:22.382342+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:46:06.138239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.370912+0000","last_change":"2023-06-28T20:07:20.381529+0000","last_active":"2023-06-28T20:07:21.370912+0000","last_peered":"2023-06-28T20:07:21.370912+0000","last_clean":"2023-06-28T20:07:21.370912+0000","last_became_active":"2023-06-28T20:07:20.381191+0000","last_became_peered":"2023-06-28T20:07:20.381191+0000","last_unstale":"2023-06-28T20:07:21.370912+0000","last_undegraded":"2023-06-28T20:07:21.370912+0000","last_fullsized":"2023-06-28T20:07:21.370912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:58:14.638684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.396345+0000","last_change":"2023-06-28T20:07:21.377519+0000","last_active":"2023-06-28T20:07:21.396345+0000","last_peered":"2023-06-28T20:07:21.396345+0000","last_clean":"2023-06-28T20:07:21.396345+0000","last_became_active":"2023-06-28T20:07:21.377225+0000","last_became_peered":"2023-06-28T20:07:21.377225+0000","last_unstale":"2023-06-28T20:07:21.396345+0000","last_undegraded":"2023-06-28T20:07:21.396345+0000","last_fullsized":"2023-06-28T20:07:21.396345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:30:40.906018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.394157+0000","last_change":"2023-06-28T20:07:20.382678+0000","last_active":"2023-06-28T20:07:21.394157+0000","last_peered":"2023-06-28T20:07:21.394157+0000","last_clean":"2023-06-28T20:07:21.394157+0000","last_became_active":"2023-06-28T20:07:20.382361+0000","last_became_peered":"2023-06-28T20:07:20.382361+0000","last_unstale":"2023-06-28T20:07:21.394157+0000","last_undegraded":"2023-06-28T20:07:21.394157+0000","last_fullsized":"2023-06-28T20:07:21.394157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:06:52.370421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381381+0000","last_change":"2023-06-28T20:07:21.397232+0000","last_active":"2023-06-28T20:07:22.381381+0000","last_peered":"2023-06-28T20:07:22.381381+0000","last_clean":"2023-06-28T20:07:22.381381+0000","last_became_active":"2023-06-28T20:07:21.396862+0000","last_became_peered":"2023-06-28T20:07:21.396862+0000","last_unstale":"2023-06-28T20:07:22.381381+0000","last_undegraded":"2023-06-28T20:07:22.381381+0000","last_fullsized":"2023-06-28T20:07:22.381381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:48:55.783926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.381592+0000","last_change":"2023-06-28T20:07:20.380628+0000","last_active":"2023-06-28T20:07:21.381592+0000","last_peered":"2023-06-28T20:07:21.381592+0000","last_clean":"2023-06-28T20:07:21.381592+0000","last_became_active":"2023-06-28T20:07:20.380033+0000","last_became_peered":"2023-06-28T20:07:20.380033+0000","last_unstale":"2023-06-28T20:07:21.381592+0000","last_undegraded":"2023-06-28T20:07:21.381592+0000","last_fullsized":"2023-06-28T20:07:21.381592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:48:54.778445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.379245+0000","last_change":"2023-06-28T20:07:21.374666+0000","last_active":"2023-06-28T20:07:21.379245+0000","last_peered":"2023-06-28T20:07:21.379245+0000","last_clean":"2023-06-28T20:07:21.379245+0000","last_became_active":"2023-06-28T20:07:21.374330+0000","last_became_peered":"2023-06-28T20:07:21.374330+0000","last_unstale":"2023-06-28T20:07:21.379245+0000","last_undegraded":"2023-06-28T20:07:21.379245+0000","last_fullsized":"2023-06-28T20:07:21.379245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:39:10.624775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.381587+0000","last_change":"2023-06-28T20:07:20.378725+0000","last_active":"2023-06-28T20:07:21.381587+0000","last_peered":"2023-06-28T20:07:21.381587+0000","last_clean":"2023-06-28T20:07:21.381587+0000","last_became_active":"2023-06-28T20:07:20.378445+0000","last_became_peered":"2023-06-28T20:07:20.378445+0000","last_unstale":"2023-06-28T20:07:21.381587+0000","last_undegraded":"2023-06-28T20:07:21.381587+0000","last_fullsized":"2023-06-28T20:07:21.381587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:06:52.370421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.378602+0000","last_change":"2023-06-28T20:07:21.374603+0000","last_active":"2023-06-28T20:07:21.378602+0000","last_peered":"2023-06-28T20:07:21.378602+0000","last_clean":"2023-06-28T20:07:21.378602+0000","last_became_active":"2023-06-28T20:07:21.374276+0000","last_became_peered":"2023-06-28T20:07:21.374276+0000","last_unstale":"2023-06-28T20:07:21.378602+0000","last_undegraded":"2023-06-28T20:07:21.378602+0000","last_fullsized":"2023-06-28T20:07:21.378602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:30:40.906018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.393258+0000","last_change":"2023-06-28T20:07:20.382290+0000","last_active":"2023-06-28T20:07:21.393258+0000","last_peered":"2023-06-28T20:07:21.393258+0000","last_clean":"2023-06-28T20:07:21.393258+0000","last_became_active":"2023-06-28T20:07:20.381991+0000","last_became_peered":"2023-06-28T20:07:20.381991+0000","last_unstale":"2023-06-28T20:07:21.393258+0000","last_undegraded":"2023-06-28T20:07:21.393258+0000","last_fullsized":"2023-06-28T20:07:21.393258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:48:02.197845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.394677+0000","last_change":"2023-06-28T20:07:21.391299+0000","last_active":"2023-06-28T20:07:21.394677+0000","last_peered":"2023-06-28T20:07:21.394677+0000","last_clean":"2023-06-28T20:07:21.394677+0000","last_became_active":"2023-06-28T20:07:21.390957+0000","last_became_peered":"2023-06-28T20:07:21.390957+0000","last_unstale":"2023-06-28T20:07:21.394677+0000","last_undegraded":"2023-06-28T20:07:21.394677+0000","last_fullsized":"2023-06-28T20:07:21.394677+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:05:46.616253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.392610+0000","last_change":"2023-06-28T20:07:20.378108+0000","last_active":"2023-06-28T20:07:21.392610+0000","last_peered":"2023-06-28T20:07:21.392610+0000","last_clean":"2023-06-28T20:07:21.392610+0000","last_became_active":"2023-06-28T20:07:20.377812+0000","last_became_peered":"2023-06-28T20:07:20.377812+0000","last_unstale":"2023-06-28T20:07:21.392610+0000","last_undegraded":"2023-06-28T20:07:21.392610+0000","last_fullsized":"2023-06-28T20:07:21.392610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:58:14.638684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.389928+0000","last_change":"2023-06-28T20:07:21.384739+0000","last_active":"2023-06-28T20:07:21.389928+0000","last_peered":"2023-06-28T20:07:21.389928+0000","last_clean":"2023-06-28T20:07:21.389928+0000","last_became_active":"2023-06-28T20:07:21.384136+0000","last_became_peered":"2023-06-28T20:07:21.384136+0000","last_unstale":"2023-06-28T20:07:21.389928+0000","last_undegraded":"2023-06-28T20:07:21.389928+0000","last_fullsized":"2023-06-28T20:07:21.389928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:30:55.763718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382228+0000","last_change":"2023-06-28T20:07:21.386188+0000","last_active":"2023-06-28T20:07:22.382228+0000","last_peered":"2023-06-28T20:07:22.382228+0000","last_clean":"2023-06-28T20:07:22.382228+0000","last_became_active":"2023-06-28T20:07:21.385171+0000","last_became_peered":"2023-06-28T20:07:21.385171+0000","last_unstale":"2023-06-28T20:07:22.382228+0000","last_undegraded":"2023-06-28T20:07:22.382228+0000","last_fullsized":"2023-06-28T20:07:22.382228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:43:18.447196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.402926+0000","last_change":"2023-06-28T20:07:21.402344+0000","last_active":"2023-06-28T20:07:21.402926+0000","last_peered":"2023-06-28T20:07:21.402926+0000","last_clean":"2023-06-28T20:07:21.402926+0000","last_became_active":"2023-06-28T20:07:21.402005+0000","last_became_peered":"2023-06-28T20:07:21.402005+0000","last_unstale":"2023-06-28T20:07:21.402926+0000","last_undegraded":"2023-06-28T20:07:21.402926+0000","last_fullsized":"2023-06-28T20:07:21.402926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:16:04.779207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.389900+0000","last_change":"2023-06-28T20:07:21.384643+0000","last_active":"2023-06-28T20:07:21.389900+0000","last_peered":"2023-06-28T20:07:21.389900+0000","last_clean":"2023-06-28T20:07:21.389900+0000","last_became_active":"2023-06-28T20:07:21.384232+0000","last_became_peered":"2023-06-28T20:07:21.384232+0000","last_unstale":"2023-06-28T20:07:21.389900+0000","last_undegraded":"2023-06-28T20:07:21.389900+0000","last_fullsized":"2023-06-28T20:07:21.389900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:06:53.375903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.394714+0000","last_change":"2023-06-28T20:07:21.391594+0000","last_active":"2023-06-28T20:07:21.394714+0000","last_peered":"2023-06-28T20:07:21.394714+0000","last_clean":"2023-06-28T20:07:21.394714+0000","last_became_active":"2023-06-28T20:07:21.391222+0000","last_became_peered":"2023-06-28T20:07:21.391222+0000","last_unstale":"2023-06-28T20:07:21.394714+0000","last_undegraded":"2023-06-28T20:07:21.394714+0000","last_fullsized":"2023-06-28T20:07:21.394714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:43:18.447196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-28T20:07:21.396508+0000","last_change":"2023-06-28T20:07:21.378640+0000","last_active":"2023-06-28T20:07:21.396508+0000","last_peered":"2023-06-28T20:07:21.396508+0000","last_clean":"2023-06-28T20:07:21.396508+0000","last_became_active":"2023-06-28T20:07:21.378345+0000","last_became_peered":"2023-06-28T20:07:21.378345+0000","last_unstale":"2023-06-28T20:07:21.396508+0000","last_undegraded":"2023-06-28T20:07:21.396508+0000","last_fullsized":"2023-06-28T20:07:21.396508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:30:55.763718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":839680,"data_stored":671247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":79099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":798720,"data_stored":662814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":72067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":72446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":208896,"data_stored":72536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":225280,"data_stored":74066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":212992,"data_stored":73098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-28T20:07:25.971 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-06-28T20:07:25.972 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-06-28T20:07:25.972 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-06-28T20:07:25.973 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-06-28T20:07:25.973 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-06-28T20:07:26.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.120+0000 7f21b986f700 1 Processor -- start 2023-06-28T20:07:26.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.120+0000 7f21b986f700 1 -- start start 2023-06-28T20:07:26.125 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.121+0000 7f21b986f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f21b40d10f0 0x7f21b40ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.121+0000 7f21b986f700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21b40cbab0 0x7f21b40b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.121+0000 7f21b986f700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21b415ab50 con 0x7f21b40cedf0 2023-06-28T20:07:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.121+0000 7f21b986f700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21b40cbf30 con 0x7f21b40d10f0 2023-06-28T20:07:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.121+0000 7f21b986f700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b40cf260 con 0x7f21b40cbab0 2023-06-28T20:07:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.121+0000 7f21b2ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f21b40d10f0 0x7f21b40ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.121+0000 7f21b2ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f21b40d10f0 0x7f21b40ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52944/0 (socket says 172.21.15.43:52944) 2023-06-28T20:07:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.121+0000 7f21b2ffd700 1 -- 172.21.15.43:0/4021599914 learned_addr learned my addr 172.21.15.43:0/4021599914 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.122+0000 7f21b37fe700 1 --2- 172.21.15.43:0/4021599914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21b40cbab0 0x7f21b40b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.122+0000 7f21b2ffd700 1 -- 172.21.15.43:0/4021599914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21b40cbab0 msgr2=0x7f21b40b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.122+0000 7f21b2ffd700 1 --2- 172.21.15.43:0/4021599914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21b40cbab0 0x7f21b40b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.122+0000 7f21b2ffd700 1 -- 172.21.15.43:0/4021599914 >> v1:172.21.15.178:6789/0 conn(0x7f21b40cedf0 legacy=0x7f21b40cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.122+0000 7f21b2ffd700 1 -- 172.21.15.43:0/4021599914 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b40cd1b0 con 0x7f21b40d10f0 2023-06-28T20:07:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.122+0000 7f21b2ffd700 1 --2- 172.21.15.43:0/4021599914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21b40d10f0 0x7f21b40ce8b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f21a80098f0 tx=0x7f21a8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2aaaf9e3d71fad server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.122+0000 7f21b1ffb700 1 -- 172.21.15.43:0/4021599914 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a8005e30 con 0x7f21b40d10f0 2023-06-28T20:07:26.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.122+0000 7f21b1ffb700 1 -- 172.21.15.43:0/4021599914 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a8006030 con 0x7f21b40d10f0 2023-06-28T20:07:26.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.122+0000 7f21b1ffb700 1 -- 172.21.15.43:0/4021599914 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a80063c0 con 0x7f21b40d10f0 2023-06-28T20:07:26.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.123+0000 7f21b986f700 1 -- 172.21.15.43:0/4021599914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21b40d10f0 msgr2=0x7f21b40ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:26.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.123+0000 7f21b986f700 1 --2- 172.21.15.43:0/4021599914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21b40d10f0 0x7f21b40ce8b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f21a80098f0 tx=0x7f21a8009c00 comp rx=0 tx=0).stop 2023-06-28T20:07:26.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.123+0000 7f21b986f700 1 -- 172.21.15.43:0/4021599914 shutdown_connections 2023-06-28T20:07:26.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.123+0000 7f21b986f700 1 --2- 172.21.15.43:0/4021599914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21b40d10f0 0x7f21b40ce8b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:26.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.123+0000 7f21b986f700 1 --2- 172.21.15.43:0/4021599914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21b40cbab0 0x7f21b40b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:26.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.123+0000 7f21b986f700 1 -- 172.21.15.43:0/4021599914 >> 172.21.15.43:0/4021599914 conn(0x7f21b40664b0 msgr2=0x7f21b409a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:26.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.123+0000 7f21b986f700 1 -- 172.21.15.43:0/4021599914 shutdown_connections 2023-06-28T20:07:26.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.123+0000 7f21b986f700 1 -- 172.21.15.43:0/4021599914 wait complete. 2023-06-28T20:07:26.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.124+0000 7f21b986f700 1 Processor -- start 2023-06-28T20:07:26.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.124+0000 7f21b986f700 1 -- start start 2023-06-28T20:07:26.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.125+0000 7f21b986f700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21b40cbab0 0x7f21b4159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:26.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.125+0000 7f21b986f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21b40cedf0 0x7f21b41e47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:26.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.125+0000 7f21b986f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21b40d10f0 0x7f21b40b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:26.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.125+0000 7f21b986f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b4159870 con 0x7f21b40d10f0 2023-06-28T20:07:26.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.125+0000 7f21b986f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b41e29a0 con 0x7f21b40cedf0 2023-06-28T20:07:26.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.125+0000 7f21b986f700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b41e2c80 con 0x7f21b40cbab0 2023-06-28T20:07:26.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.125+0000 7f21b37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21b40d10f0 0x7f21b40b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:26.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.125+0000 7f21b37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21b40d10f0 0x7f21b40b08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52954/0 (socket says 172.21.15.43:52954) 2023-06-28T20:07:26.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.125+0000 7f21b37fe700 1 -- 172.21.15.43:0/957411290 learned_addr learned my addr 172.21.15.43:0/957411290 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:26.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.125+0000 7f21b2ffd700 1 --2- 172.21.15.43:0/957411290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21b40cbab0 0x7f21b4159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:26.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.125+0000 7f21b27fc700 1 --2- 172.21.15.43:0/957411290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21b40cedf0 0x7f21b41e47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:26.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.125+0000 7f21b37fe700 1 -- 172.21.15.43:0/957411290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21b40cbab0 msgr2=0x7f21b4159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:26.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.125+0000 7f21b37fe700 1 --2- 172.21.15.43:0/957411290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21b40cbab0 0x7f21b4159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:26.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.125+0000 7f21b37fe700 1 -- 172.21.15.43:0/957411290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21b40cedf0 msgr2=0x7f21b41e47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:26.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.125+0000 7f21b37fe700 1 --2- 172.21.15.43:0/957411290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21b40cedf0 0x7f21b41e47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:26.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.125+0000 7f21b37fe700 1 -- 172.21.15.43:0/957411290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a8009500 con 0x7f21b40d10f0 2023-06-28T20:07:26.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.125+0000 7f21b37fe700 1 --2- 172.21.15.43:0/957411290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21b40d10f0 0x7f21b40b08c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00cd70 tx=0x7f21ac00a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:26.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.126+0000 7f219bfff700 1 -- 172.21.15.43:0/957411290 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac003a20 con 0x7f21b40d10f0 2023-06-28T20:07:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.126+0000 7f219bfff700 1 -- 172.21.15.43:0/957411290 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21ac003b80 con 0x7f21b40d10f0 2023-06-28T20:07:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.126+0000 7f21b986f700 1 -- 172.21.15.43:0/957411290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b41e2f60 con 0x7f21b40d10f0 2023-06-28T20:07:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.126+0000 7f219bfff700 1 -- 172.21.15.43:0/957411290 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac004020 con 0x7f21b40d10f0 2023-06-28T20:07:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.126+0000 7f21b986f700 1 -- 172.21.15.43:0/957411290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b406b0a0 con 0x7f21b40d10f0 2023-06-28T20:07:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.126+0000 7f21b986f700 1 -- 172.21.15.43:0/957411290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b406b260 con 0x7f21b40d10f0 2023-06-28T20:07:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.127+0000 7f219bfff700 1 -- 172.21.15.43:0/957411290 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f21ac004180 con 0x7f21b40d10f0 2023-06-28T20:07:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.127+0000 7f219bfff700 1 --2- 172.21.15.43:0/957411290 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f21a006ace0 0x7f21a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.128+0000 7f219bfff700 1 -- 172.21.15.43:0/957411290 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f21ac012070 con 0x7f21b40d10f0 2023-06-28T20:07:26.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.128+0000 7f21b2ffd700 1 --2- 172.21.15.43:0/957411290 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f21a006ace0 0x7f21a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:26.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.128+0000 7f21b2ffd700 1 --2- 172.21.15.43:0/957411290 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f21a006ace0 0x7f21a006d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f21a8000c00 tx=0x7f21a8008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:26.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.132+0000 7f219bfff700 1 -- 172.21.15.43:0/957411290 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f21ac054c50 con 0x7f21b40d10f0 2023-06-28T20:07:26.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.388+0000 7f21b986f700 1 -- 172.21.15.43:0/957411290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f21b409a570 con 0x7f21b40d10f0 2023-06-28T20:07:26.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.391+0000 7f219bfff700 1 -- 172.21.15.43:0/957411290 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170837 (secure 0 0 0) 0x7f21ac058270 con 0x7f21b40d10f0 2023-06-28T20:07:26.396 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:26.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.394+0000 7f21b986f700 1 -- 172.21.15.43:0/957411290 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f21a006ace0 msgr2=0x7f21a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:26.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.394+0000 7f21b986f700 1 --2- 172.21.15.43:0/957411290 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f21a006ace0 0x7f21a006d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f21a8000c00 tx=0x7f21a8008070 comp rx=0 tx=0).stop 2023-06-28T20:07:26.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.394+0000 7f21b986f700 1 -- 172.21.15.43:0/957411290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21b40d10f0 msgr2=0x7f21b40b08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:26.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.394+0000 7f21b986f700 1 --2- 172.21.15.43:0/957411290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21b40d10f0 0x7f21b40b08c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00cd70 tx=0x7f21ac00a570 comp rx=0 tx=0).stop 2023-06-28T20:07:26.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.394+0000 7f21b986f700 1 -- 172.21.15.43:0/957411290 shutdown_connections 2023-06-28T20:07:26.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.394+0000 7f21b986f700 1 --2- 172.21.15.43:0/957411290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21b40d10f0 0x7f21b40b08c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:26.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.394+0000 7f21b986f700 1 --2- 172.21.15.43:0/957411290 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f21a006ace0 0x7f21a006d190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:26.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.394+0000 7f21b986f700 1 --2- 172.21.15.43:0/957411290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f21b40cbab0 0x7f21b4159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:26.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.394+0000 7f21b986f700 1 --2- 172.21.15.43:0/957411290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21b40cedf0 0x7f21b41e47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:26.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.394+0000 7f21b986f700 1 -- 172.21.15.43:0/957411290 >> 172.21.15.43:0/957411290 conn(0x7f21b40664b0 msgr2=0x7f21b415a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:26.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.394+0000 7f21b986f700 1 -- 172.21.15.43:0/957411290 shutdown_connections 2023-06-28T20:07:26.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.394+0000 7f21b986f700 1 -- 172.21.15.43:0/957411290 wait complete. 2023-06-28T20:07:26.410 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":5,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":406144153},{"type":"v1","addr":"172.21.15.43:6833","nonce":406144153}]},"active_addr":"172.21.15.43:6833/406144153","active_change":"2023-06-28T20:07:07.974736+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4126,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":2515703897}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":696398376}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":2655254485}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":730188937}]}]} 2023-06-28T20:07:26.418 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-06-28T20:07:26.419 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-06-28T20:07:26.419 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:07:26.450 INFO:tasks.ceph.mds.c.smithi043.stderr:2023-06-28T20:07:26.447+0000 7ff981b9e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-28T20:07:26.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.541+0000 7fda352a4700 1 Processor -- start 2023-06-28T20:07:26.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.541+0000 7fda352a4700 1 -- start start 2023-06-28T20:07:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.542+0000 7fda352a4700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fda300b2170 0x7fda300d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.542+0000 7fda352a4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fda300d2b00 0x7fda3006a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.542+0000 7fda352a4700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda30160270 con 0x7fda300cf120 2023-06-28T20:07:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.542+0000 7fda352a4700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda300d0910 con 0x7fda300d2b00 2023-06-28T20:07:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.542+0000 7fda352a4700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda300cf870 con 0x7fda300b2170 2023-06-28T20:07:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.542+0000 7fda2f7fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fda300d2b00 0x7fda3006a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.542+0000 7fda2f7fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fda300d2b00 0x7fda3006a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52962/0 (socket says 172.21.15.43:52962) 2023-06-28T20:07:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.542+0000 7fda2f7fe700 1 -- 172.21.15.43:0/1721214017 learned_addr learned my addr 172.21.15.43:0/1721214017 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.542+0000 7fda2effd700 1 --2- 172.21.15.43:0/1721214017 >> v2:172.21.15.178:3300/0 conn(0x7fda300b2170 0x7fda300d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.542+0000 7fda2f7fe700 1 -- 172.21.15.43:0/1721214017 >> v2:172.21.15.178:3300/0 conn(0x7fda300b2170 msgr2=0x7fda300d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.542+0000 7fda2f7fe700 1 --2- 172.21.15.43:0/1721214017 >> v2:172.21.15.178:3300/0 conn(0x7fda300b2170 0x7fda300d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:26.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.542+0000 7fda2f7fe700 1 -- 172.21.15.43:0/1721214017 >> v1:172.21.15.178:6789/0 conn(0x7fda300cf120 legacy=0x7fda300b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.542+0000 7fda2f7fe700 1 -- 172.21.15.43:0/1721214017 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda300cf590 con 0x7fda300d2b00 2023-06-28T20:07:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.543+0000 7fda2f7fe700 1 --2- 172.21.15.43:0/1721214017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda300d2b00 0x7fda3006a7b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fda3008c2d0 tx=0x7fda2800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee77e7384a05b4c server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.543+0000 7fda2dffb700 1 -- 172.21.15.43:0/1721214017 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2800e070 con 0x7fda300d2b00 2023-06-28T20:07:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.543+0000 7fda2dffb700 1 -- 172.21.15.43:0/1721214017 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2800b980 con 0x7fda300d2b00 2023-06-28T20:07:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.543+0000 7fda2dffb700 1 -- 172.21.15.43:0/1721214017 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2800bd30 con 0x7fda300d2b00 2023-06-28T20:07:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.543+0000 7fda352a4700 1 -- 172.21.15.43:0/1721214017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda300d2b00 msgr2=0x7fda3006a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.543+0000 7fda352a4700 1 --2- 172.21.15.43:0/1721214017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda300d2b00 0x7fda3006a7b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fda3008c2d0 tx=0x7fda2800b330 comp rx=0 tx=0).stop 2023-06-28T20:07:26.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.543+0000 7fda352a4700 1 -- 172.21.15.43:0/1721214017 shutdown_connections 2023-06-28T20:07:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.543+0000 7fda352a4700 1 --2- 172.21.15.43:0/1721214017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda300d2b00 0x7fda3006a7b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.543+0000 7fda352a4700 1 --2- 172.21.15.43:0/1721214017 >> v2:172.21.15.178:3300/0 conn(0x7fda300b2170 0x7fda300d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.543+0000 7fda352a4700 1 -- 172.21.15.43:0/1721214017 >> 172.21.15.43:0/1721214017 conn(0x7fda3006c1e0 msgr2=0x7fda3006c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:26.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.543+0000 7fda352a4700 1 -- 172.21.15.43:0/1721214017 shutdown_connections 2023-06-28T20:07:26.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.543+0000 7fda352a4700 1 -- 172.21.15.43:0/1721214017 wait complete. 2023-06-28T20:07:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.545+0000 7fda352a4700 1 Processor -- start 2023-06-28T20:07:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.545+0000 7fda352a4700 1 -- start start 2023-06-28T20:07:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.546+0000 7fda352a4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda300b2170 0x7fda3014dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.546+0000 7fda352a4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda300cf120 0x7fda301f2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:26.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.546+0000 7fda352a4700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda300b7710 0x7fda300b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.546+0000 7fda352a4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda300b80c0 con 0x7fda300cf120 2023-06-28T20:07:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.546+0000 7fda352a4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3015ce80 con 0x7fda300b2170 2023-06-28T20:07:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.546+0000 7fda352a4700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3015d160 con 0x7fda300b7710 2023-06-28T20:07:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.546+0000 7fda2e7fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda300cf120 0x7fda301f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.546+0000 7fda2e7fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda300cf120 0x7fda301f2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52966/0 (socket says 172.21.15.43:52966) 2023-06-28T20:07:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.546+0000 7fda2e7fc700 1 -- 172.21.15.43:0/2368291810 learned_addr learned my addr 172.21.15.43:0/2368291810 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.546+0000 7fda2f7fe700 1 --2- 172.21.15.43:0/2368291810 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda300b7710 0x7fda300b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:26.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.546+0000 7fda2effd700 1 --2- 172.21.15.43:0/2368291810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda300b2170 0x7fda3014dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.546+0000 7fda2e7fc700 1 -- 172.21.15.43:0/2368291810 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda300b7710 msgr2=0x7fda300b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.546+0000 7fda2e7fc700 1 --2- 172.21.15.43:0/2368291810 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda300b7710 0x7fda300b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.546+0000 7fda2e7fc700 1 -- 172.21.15.43:0/2368291810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda300b2170 msgr2=0x7fda3014dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.546+0000 7fda2e7fc700 1 --2- 172.21.15.43:0/2368291810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda300b2170 0x7fda3014dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.546+0000 7fda2e7fc700 1 -- 172.21.15.43:0/2368291810 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda28009b30 con 0x7fda300cf120 2023-06-28T20:07:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.547+0000 7fda2e7fc700 1 --2- 172.21.15.43:0/2368291810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda300cf120 0x7fda301f2370 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fda2000c500 tx=0x7fda2000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:26.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.547+0000 7fda177fe700 1 -- 172.21.15.43:0/2368291810 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda200052d0 con 0x7fda300cf120 2023-06-28T20:07:26.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.547+0000 7fda177fe700 1 -- 172.21.15.43:0/2368291810 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda20005430 con 0x7fda300cf120 2023-06-28T20:07:26.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.547+0000 7fda177fe700 1 -- 172.21.15.43:0/2368291810 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20006aa0 con 0x7fda300cf120 2023-06-28T20:07:26.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.547+0000 7fda352a4700 1 -- 172.21.15.43:0/2368291810 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda3015d440 con 0x7fda300cf120 2023-06-28T20:07:26.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.547+0000 7fda352a4700 1 -- 172.21.15.43:0/2368291810 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda3015da00 con 0x7fda300cf120 2023-06-28T20:07:26.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.548+0000 7fda352a4700 1 -- 172.21.15.43:0/2368291810 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda300728b0 con 0x7fda300cf120 2023-06-28T20:07:26.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.548+0000 7fda177fe700 1 -- 172.21.15.43:0/2368291810 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fda20003680 con 0x7fda300cf120 2023-06-28T20:07:26.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.549+0000 7fda177fe700 1 --2- 172.21.15.43:0/2368291810 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fda1c06ace0 0x7fda1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:26.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.549+0000 7fda177fe700 1 -- 172.21.15.43:0/2368291810 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fda200885d0 con 0x7fda300cf120 2023-06-28T20:07:26.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.549+0000 7fda2effd700 1 --2- 172.21.15.43:0/2368291810 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fda1c06ace0 0x7fda1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:26.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.549+0000 7fda2effd700 1 --2- 172.21.15.43:0/2368291810 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fda1c06ace0 0x7fda1c06d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fda24006b20 tx=0x7fda24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:26.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.553+0000 7fda177fe700 1 -- 172.21.15.43:0/2368291810 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fda20053680 con 0x7fda300cf120 2023-06-28T20:07:26.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.755+0000 7fda352a4700 1 -- 172.21.15.43:0/2368291810 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda300b5320 con 0x7fda300cf120 2023-06-28T20:07:26.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.756+0000 7fda177fe700 1 -- 172.21.15.43:0/2368291810 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18221 (secure 0 0 0) 0x7fda20056ca0 con 0x7fda300cf120 2023-06-28T20:07:26.760 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:26.760 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:07:22.377248+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:07:18.624759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-28T20:07:19.760017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:07:26.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.758+0000 7fda352a4700 1 -- 172.21.15.43:0/2368291810 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fda1c06ace0 msgr2=0x7fda1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:26.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.758+0000 7fda352a4700 1 --2- 172.21.15.43:0/2368291810 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fda1c06ace0 0x7fda1c06d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fda24006b20 tx=0x7fda24008040 comp rx=0 tx=0).stop 2023-06-28T20:07:26.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.758+0000 7fda352a4700 1 -- 172.21.15.43:0/2368291810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda300cf120 msgr2=0x7fda301f2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:26.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.758+0000 7fda352a4700 1 --2- 172.21.15.43:0/2368291810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda300cf120 0x7fda301f2370 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fda2000c500 tx=0x7fda2000c810 comp rx=0 tx=0).stop 2023-06-28T20:07:26.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.758+0000 7fda352a4700 1 -- 172.21.15.43:0/2368291810 shutdown_connections 2023-06-28T20:07:26.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.758+0000 7fda352a4700 1 --2- 172.21.15.43:0/2368291810 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fda1c06ace0 0x7fda1c06d190 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:26.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.758+0000 7fda352a4700 1 --2- 172.21.15.43:0/2368291810 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda300b7710 0x7fda300b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:26.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.758+0000 7fda352a4700 1 --2- 172.21.15.43:0/2368291810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda300b2170 0x7fda3014dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:26.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.758+0000 7fda352a4700 1 --2- 172.21.15.43:0/2368291810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda300cf120 0x7fda301f2370 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:26.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.759+0000 7fda352a4700 1 -- 172.21.15.43:0/2368291810 >> 172.21.15.43:0/2368291810 conn(0x7fda3006c1e0 msgr2=0x7fda300b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:26.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.759+0000 7fda352a4700 1 -- 172.21.15.43:0/2368291810 shutdown_connections 2023-06-28T20:07:26.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.759+0000 7fda352a4700 1 -- 172.21.15.43:0/2368291810 wait complete. 2023-06-28T20:07:26.774 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-06-28T20:07:26.774 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:07:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.937+0000 7efdb91a1700 1 Processor -- start 2023-06-28T20:07:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.937+0000 7efdb91a1700 1 -- start start 2023-06-28T20:07:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.938+0000 7efdb91a1700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdb40cc460 0x7efdb40cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:26.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.938+0000 7efdb91a1700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdb40ccf30 con 0x7efdb406a0d0 2023-06-28T20:07:26.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.938+0000 7efdb91a1700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdb40cd210 con 0x7efdb406a4a0 2023-06-28T20:07:26.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.938+0000 7efdb91a1700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb4142d80 con 0x7efdb40cc460 2023-06-28T20:07:26.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.938+0000 7efdb3fff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7efdb406a0d0 0x7efdb40ca130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42158/0 (socket says 172.21.15.43:42158) 2023-06-28T20:07:26.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.938+0000 7efdb3fff700 1 -- 172.21.15.43:0/1602098123 learned_addr learned my addr 172.21.15.43:0/1602098123 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:26.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.938+0000 7efdb89a0700 1 --2- 172.21.15.43:0/1602098123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdb40cc460 0x7efdb40cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:26.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.938+0000 7efdb2ffd700 1 -- 172.21.15.43:0/1602098123 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941019151 0 0) 0x7efdb40ccf30 con 0x7efdb406a0d0 2023-06-28T20:07:26.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.938+0000 7efdb2ffd700 1 -- 172.21.15.43:0/1602098123 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efda00035b0 con 0x7efdb406a0d0 2023-06-28T20:07:26.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.938+0000 7efdb2ffd700 1 -- 172.21.15.43:0/1602098123 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 54992579 0 0) 0x7efdb40cd210 con 0x7efdb406a4a0 2023-06-28T20:07:26.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.938+0000 7efdb2ffd700 1 -- 172.21.15.43:0/1602098123 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdb40ccf30 con 0x7efdb406a4a0 2023-06-28T20:07:26.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.938+0000 7efdb89a0700 1 -- 172.21.15.43:0/1602098123 >> v1:172.21.15.178:6789/0 conn(0x7efdb406a4a0 legacy=0x7efdb4142670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:26.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.938+0000 7efdb89a0700 1 -- 172.21.15.43:0/1602098123 >> v1:172.21.15.43:6789/0 conn(0x7efdb406a0d0 legacy=0x7efdb40ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:26.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.938+0000 7efdb89a0700 1 -- 172.21.15.43:0/1602098123 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb40709f0 con 0x7efdb40cc460 2023-06-28T20:07:26.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.939+0000 7efdb89a0700 1 --2- 172.21.15.43:0/1602098123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdb40cc460 0x7efdb40cc8c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efdac009dc0 tx=0x7efdac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be02d6fc60d82a95 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:26.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.939+0000 7efdb2ffd700 1 -- 172.21.15.43:0/1602098123 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac00e070 con 0x7efdb40cc460 2023-06-28T20:07:26.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.939+0000 7efdb2ffd700 1 -- 172.21.15.43:0/1602098123 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdac00bb20 con 0x7efdb40cc460 2023-06-28T20:07:26.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.939+0000 7efdb2ffd700 1 -- 172.21.15.43:0/1602098123 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac012430 con 0x7efdb40cc460 2023-06-28T20:07:26.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.939+0000 7efdb91a1700 1 -- 172.21.15.43:0/1602098123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdb40cc460 msgr2=0x7efdb40cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:26.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.939+0000 7efdb91a1700 1 --2- 172.21.15.43:0/1602098123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdb40cc460 0x7efdb40cc8c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efdac009dc0 tx=0x7efdac00b330 comp rx=0 tx=0).stop 2023-06-28T20:07:26.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.939+0000 7efdb91a1700 1 -- 172.21.15.43:0/1602098123 shutdown_connections 2023-06-28T20:07:26.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.939+0000 7efdb91a1700 1 --2- 172.21.15.43:0/1602098123 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdb40cc460 0x7efdb40cc8c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:26.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.939+0000 7efdb91a1700 1 -- 172.21.15.43:0/1602098123 >> 172.21.15.43:0/1602098123 conn(0x7efdb40b1790 msgr2=0x7efdb40b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:26.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.939+0000 7efdb91a1700 1 -- 172.21.15.43:0/1602098123 shutdown_connections 2023-06-28T20:07:26.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.939+0000 7efdb91a1700 1 -- 172.21.15.43:0/1602098123 wait complete. 2023-06-28T20:07:26.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.941+0000 7efdb91a1700 1 Processor -- start 2023-06-28T20:07:26.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.941+0000 7efdb91a1700 1 -- start start 2023-06-28T20:07:26.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.941+0000 7efdb91a1700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdb406a0d0 0x7efdb4145590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:26.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.941+0000 7efdb91a1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdb406a4a0 0x7efdb40b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:26.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.941+0000 7efdb91a1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb40cc460 0x7efdb40b4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:26.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.941+0000 7efdb91a1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb40b0220 con 0x7efdb40cc460 2023-06-28T20:07:26.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.941+0000 7efdb91a1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb40b0500 con 0x7efdb406a4a0 2023-06-28T20:07:26.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.941+0000 7efdb91a1700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb40b07e0 con 0x7efdb406a0d0 2023-06-28T20:07:26.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.942+0000 7efdb89a0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb40cc460 0x7efdb40b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:26.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.942+0000 7efdb89a0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb40cc460 0x7efdb40b4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52982/0 (socket says 172.21.15.43:52982) 2023-06-28T20:07:26.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.942+0000 7efdb89a0700 1 -- 172.21.15.43:0/2471017119 learned_addr learned my addr 172.21.15.43:0/2471017119 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:26.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.942+0000 7efdb3fff700 1 --2- 172.21.15.43:0/2471017119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdb406a0d0 0x7efdb4145590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:26.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.942+0000 7efdb37fe700 1 --2- 172.21.15.43:0/2471017119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdb406a4a0 0x7efdb40b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:26.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.942+0000 7efdb89a0700 1 -- 172.21.15.43:0/2471017119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdb406a0d0 msgr2=0x7efdb4145590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:26.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.942+0000 7efdb89a0700 1 --2- 172.21.15.43:0/2471017119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdb406a0d0 0x7efdb4145590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:26.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.942+0000 7efdb89a0700 1 -- 172.21.15.43:0/2471017119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdb406a4a0 msgr2=0x7efdb40b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:26.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.942+0000 7efdb89a0700 1 --2- 172.21.15.43:0/2471017119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdb406a4a0 0x7efdb40b40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:26.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.942+0000 7efdb89a0700 1 -- 172.21.15.43:0/2471017119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdac009a70 con 0x7efdb40cc460 2023-06-28T20:07:26.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.942+0000 7efdb89a0700 1 --2- 172.21.15.43:0/2471017119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb40cc460 0x7efdb40b4620 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efdac00b370 tx=0x7efdac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:26.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.942+0000 7efdb0ff9700 1 -- 172.21.15.43:0/2471017119 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac01b810 con 0x7efdb40cc460 2023-06-28T20:07:26.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.942+0000 7efdb0ff9700 1 -- 172.21.15.43:0/2471017119 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdac020070 con 0x7efdb40cc460 2023-06-28T20:07:26.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.942+0000 7efdb91a1700 1 -- 172.21.15.43:0/2471017119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb40b0ac0 con 0x7efdb40cc460 2023-06-28T20:07:26.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.942+0000 7efdb0ff9700 1 -- 172.21.15.43:0/2471017119 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac01bbb0 con 0x7efdb40cc460 2023-06-28T20:07:26.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.942+0000 7efdb91a1700 1 -- 172.21.15.43:0/2471017119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb41f1960 con 0x7efdb40cc460 2023-06-28T20:07:26.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.943+0000 7efdb91a1700 1 -- 172.21.15.43:0/2471017119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb40728b0 con 0x7efdb40cc460 2023-06-28T20:07:26.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.944+0000 7efdb0ff9700 1 -- 172.21.15.43:0/2471017119 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efdac01bd10 con 0x7efdb40cc460 2023-06-28T20:07:26.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.944+0000 7efdb0ff9700 1 --2- 172.21.15.43:0/2471017119 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efd9406ace0 0x7efd9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:26.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.944+0000 7efdb3fff700 1 --2- 172.21.15.43:0/2471017119 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efd9406ace0 0x7efd9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:26.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.944+0000 7efdb0ff9700 1 -- 172.21.15.43:0/2471017119 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efdac022070 con 0x7efdb40cc460 2023-06-28T20:07:26.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.944+0000 7efdb3fff700 1 --2- 172.21.15.43:0/2471017119 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efd9406ace0 0x7efd9406d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efd9c007fd0 tx=0x7efd9c007e00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:26.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:26.949+0000 7efdb0ff9700 1 -- 172.21.15.43:0/2471017119 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efdac056880 con 0x7efdb40cc460 2023-06-28T20:07:27.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.148+0000 7efdb91a1700 1 -- 172.21.15.43:0/2471017119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efdb40cf6b0 con 0x7efdb40cc460 2023-06-28T20:07:27.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.149+0000 7efdb0ff9700 1 -- 172.21.15.43:0/2471017119 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18221 (secure 0 0 0) 0x7efdac059ea0 con 0x7efdb40cc460 2023-06-28T20:07:27.152 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:27.153 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:07:22.377248+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:07:18.624759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-28T20:07:19.760017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:07:27.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.151+0000 7efdb91a1700 1 -- 172.21.15.43:0/2471017119 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efd9406ace0 msgr2=0x7efd9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.151+0000 7efdb91a1700 1 --2- 172.21.15.43:0/2471017119 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efd9406ace0 0x7efd9406d190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7efd9c007fd0 tx=0x7efd9c007e00 comp rx=0 tx=0).stop 2023-06-28T20:07:27.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.151+0000 7efdb91a1700 1 -- 172.21.15.43:0/2471017119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb40cc460 msgr2=0x7efdb40b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.151+0000 7efdb91a1700 1 --2- 172.21.15.43:0/2471017119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb40cc460 0x7efdb40b4620 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efdac00b370 tx=0x7efdac004ad0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.152+0000 7efdb91a1700 1 -- 172.21.15.43:0/2471017119 shutdown_connections 2023-06-28T20:07:27.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.152+0000 7efdb91a1700 1 --2- 172.21.15.43:0/2471017119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb40cc460 0x7efdb40b4620 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.152+0000 7efdb91a1700 1 --2- 172.21.15.43:0/2471017119 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efd9406ace0 0x7efd9406d190 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.152+0000 7efdb91a1700 1 --2- 172.21.15.43:0/2471017119 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efdb406a0d0 0x7efdb4145590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.152+0000 7efdb91a1700 1 --2- 172.21.15.43:0/2471017119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efdb406a4a0 0x7efdb40b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.152+0000 7efdb91a1700 1 -- 172.21.15.43:0/2471017119 >> 172.21.15.43:0/2471017119 conn(0x7efdb40b1790 msgr2=0x7efdb40ca110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.152+0000 7efdb91a1700 1 -- 172.21.15.43:0/2471017119 shutdown_connections 2023-06-28T20:07:27.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.152+0000 7efdb91a1700 1 -- 172.21.15.43:0/2471017119 wait complete. 2023-06-28T20:07:27.168 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-06-28T20:07:27.169 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-06-28T20:07:27.169 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-06-28T20:07:27.170 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-06-28T20:07:27.170 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-06-28T20:07:27.170 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-06-28T20:07:27.171 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-06-28T20:07:27.171 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-06-28T20:07:27.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.377+0000 7fdbd1845700 1 Processor -- start 2023-06-28T20:07:27.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.377+0000 7fdbd1845700 1 -- start start 2023-06-28T20:07:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.378+0000 7fdbd1845700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbcc1690f0 0x7fdbcc0ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.378+0000 7fdbd1845700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdbcc0ceee0 0x7fdbcc06c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.378+0000 7fdbd1845700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbcc16d6a0 con 0x7fdbcc06cbb0 2023-06-28T20:07:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.378+0000 7fdbd1845700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbcc0cf350 con 0x7fdbcc0ceee0 2023-06-28T20:07:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.378+0000 7fdbd1845700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc06d030 con 0x7fdbcc1690f0 2023-06-28T20:07:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.378+0000 7fdbcb7fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdbcc06cbb0 0x7fdbcc0cc5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42176/0 (socket says 172.21.15.43:42176) 2023-06-28T20:07:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.378+0000 7fdbca7fc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdbcc0ceee0 0x7fdbcc06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.378+0000 7fdbcb7fe700 1 -- 172.21.15.43:0/2081219911 learned_addr learned my addr 172.21.15.43:0/2081219911 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:27.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.379+0000 7fdbcaffd700 1 --2- 172.21.15.43:0/2081219911 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbcc1690f0 0x7fdbcc0ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.379+0000 7fdbcaffd700 1 -- 172.21.15.43:0/2081219911 >> v2:172.21.15.43:3300/0 conn(0x7fdbcc0ceee0 msgr2=0x7fdbcc06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.379+0000 7fdbcaffd700 1 --2- 172.21.15.43:0/2081219911 >> v2:172.21.15.43:3300/0 conn(0x7fdbcc0ceee0 0x7fdbcc06c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.379+0000 7fdbcaffd700 1 -- 172.21.15.43:0/2081219911 >> v1:172.21.15.43:6789/0 conn(0x7fdbcc06cbb0 legacy=0x7fdbcc0cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.379+0000 7fdbcaffd700 1 -- 172.21.15.43:0/2081219911 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbcc08a900 con 0x7fdbcc1690f0 2023-06-28T20:07:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.380+0000 7fdbcaffd700 1 --2- 172.21.15.43:0/2081219911 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbcc1690f0 0x7fdbcc0ce9a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400cb00 tx=0x7fdbc400a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20f6cf97d3932d67 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.380+0000 7fdbc9ffb700 1 -- 172.21.15.43:0/2081219911 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4007900 con 0x7fdbcc1690f0 2023-06-28T20:07:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.380+0000 7fdbc9ffb700 1 -- 172.21.15.43:0/2081219911 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc4007a60 con 0x7fdbcc1690f0 2023-06-28T20:07:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.380+0000 7fdbc9ffb700 1 -- 172.21.15.43:0/2081219911 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4007e10 con 0x7fdbcc1690f0 2023-06-28T20:07:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.381+0000 7fdbd1845700 1 -- 172.21.15.43:0/2081219911 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbcc1690f0 msgr2=0x7fdbcc0ce9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.381+0000 7fdbd1845700 1 --2- 172.21.15.43:0/2081219911 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbcc1690f0 0x7fdbcc0ce9a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400cb00 tx=0x7fdbc400a330 comp rx=0 tx=0).stop 2023-06-28T20:07:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.384+0000 7fdbd1845700 1 -- 172.21.15.43:0/2081219911 shutdown_connections 2023-06-28T20:07:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.384+0000 7fdbd1845700 1 --2- 172.21.15.43:0/2081219911 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbcc1690f0 0x7fdbcc0ce9a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.384+0000 7fdbd1845700 1 --2- 172.21.15.43:0/2081219911 >> v2:172.21.15.43:3300/0 conn(0x7fdbcc0ceee0 0x7fdbcc06c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.384+0000 7fdbd1845700 1 -- 172.21.15.43:0/2081219911 >> 172.21.15.43:0/2081219911 conn(0x7fdbcc069ef0 msgr2=0x7fdbcc06a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.387+0000 7efc25f20700 1 Processor -- start 2023-06-28T20:07:27.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.387+0000 7efc25f20700 1 -- start start 2023-06-28T20:07:27.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.388+0000 7efc25f20700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efc2006b130 0x7efc200d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.388+0000 7efc25f20700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc200d1db0 con 0x7efc200b2300 2023-06-28T20:07:27.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.388+0000 7efc25f20700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc200d11a0 con 0x7efc2006ad60 2023-06-28T20:07:27.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.388+0000 7efc25f20700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc200cdc40 con 0x7efc2006b130 2023-06-28T20:07:27.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.387+0000 7fdbd1845700 1 -- 172.21.15.43:0/2081219911 shutdown_connections 2023-06-28T20:07:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.389+0000 7fdbd1845700 1 -- 172.21.15.43:0/2081219911 wait complete. 2023-06-28T20:07:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.389+0000 7efc1effd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7efc200b2300 0x7efc200d2240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42180/0 (socket says 172.21.15.43:42180) 2023-06-28T20:07:27.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.389+0000 7efc1effd700 1 -- 172.21.15.43:0/1348090230 learned_addr learned my addr 172.21.15.43:0/1348090230 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:27.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.389+0000 7efc1ffff700 1 --2- 172.21.15.43:0/1348090230 >> v2:172.21.15.178:3300/0 conn(0x7efc2006b130 0x7efc200d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.389+0000 7efc1e7fc700 1 -- 172.21.15.43:0/1348090230 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477866911 0 0) 0x7efc200d11a0 con 0x7efc2006ad60 2023-06-28T20:07:27.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.389+0000 7efc1e7fc700 1 -- 172.21.15.43:0/1348090230 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc080035b0 con 0x7efc2006ad60 2023-06-28T20:07:27.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.389+0000 7efc1e7fc700 1 -- 172.21.15.43:0/1348090230 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 891395142 0 0) 0x7efc200d1db0 con 0x7efc200b2300 2023-06-28T20:07:27.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.389+0000 7efc1e7fc700 1 -- 172.21.15.43:0/1348090230 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc200d11a0 con 0x7efc200b2300 2023-06-28T20:07:27.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.389+0000 7efc1e7fc700 1 -- 172.21.15.43:0/1348090230 <== mon.1 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2843336814 0 0) 0x7efc080035b0 con 0x7efc2006ad60 2023-06-28T20:07:27.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.389+0000 7efc1e7fc700 1 -- 172.21.15.43:0/1348090230 >> v2:172.21.15.178:3300/0 conn(0x7efc2006b130 msgr2=0x7efc200d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.389+0000 7efc1e7fc700 1 --2- 172.21.15.43:0/1348090230 >> v2:172.21.15.178:3300/0 conn(0x7efc2006b130 0x7efc200d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.389+0000 7efc1e7fc700 1 -- 172.21.15.43:0/1348090230 >> v1:172.21.15.43:6789/0 conn(0x7efc200b2300 legacy=0x7efc200d2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.389+0000 7efc1e7fc700 1 -- 172.21.15.43:0/1348090230 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc200d1480 con 0x7efc2006ad60 2023-06-28T20:07:27.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.389+0000 7efc1e7fc700 1 -- 172.21.15.43:0/1348090230 <== mon.1 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7efc14001670 con 0x7efc2006ad60 2023-06-28T20:07:27.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.390+0000 7efc1e7fc700 1 -- 172.21.15.43:0/1348090230 >> v1:172.21.15.178:6789/0 conn(0x7efc2006ad60 legacy=0x7efc200cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.390+0000 7efc1e7fc700 1 --2- 172.21.15.43:0/1348090230 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc08003b90 0x7efc08003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.390+0000 7efc1e7fc700 1 --2- 172.21.15.43:0/1348090230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc08004850 0x7efc08004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.390+0000 7efc1e7fc700 1 --2- 172.21.15.43:0/1348090230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc08005200 0x7efc0800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.390+0000 7efc1e7fc700 1 -- 172.21.15.43:0/1348090230 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc200cdc40 con 0x7efc08004850 2023-06-28T20:07:27.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.390+0000 7efc1e7fc700 1 -- 172.21.15.43:0/1348090230 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0800c000 con 0x7efc08005200 2023-06-28T20:07:27.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.390+0000 7efc1e7fc700 1 -- 172.21.15.43:0/1348090230 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc0800c1c0 con 0x7efc08003b90 2023-06-28T20:07:27.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.390+0000 7fd1626ef700 1 Processor -- start 2023-06-28T20:07:27.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.390+0000 7efc1f7fe700 1 --2- 172.21.15.43:0/1348090230 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc08003b90 0x7efc08003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.390+0000 7efc1ffff700 1 --2- 172.21.15.43:0/1348090230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc08005200 0x7efc0800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.390+0000 7fdbd1845700 1 Processor -- start 2023-06-28T20:07:27.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.391+0000 7efc1effd700 1 --2- 172.21.15.43:0/1348090230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc08004850 0x7efc08004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.391+0000 7efc1effd700 1 -- 172.21.15.43:0/1348090230 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc08003b90 msgr2=0x7efc08003f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.391+0000 7efc1effd700 1 --2- 172.21.15.43:0/1348090230 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc08003b90 0x7efc08003f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.391+0000 7efc1effd700 1 -- 172.21.15.43:0/1348090230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc08005200 msgr2=0x7efc0800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.391+0000 7efc1effd700 1 --2- 172.21.15.43:0/1348090230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc08005200 0x7efc0800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.391+0000 7efc1effd700 1 -- 172.21.15.43:0/1348090230 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc200d1480 con 0x7efc08004850 2023-06-28T20:07:27.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.391+0000 7efc1effd700 1 --2- 172.21.15.43:0/1348090230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc08004850 0x7efc08004cc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efc10001a60 tx=0x7efc08005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d1fb4690486b41 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.392+0000 7efc1ffff700 1 -- 172.21.15.43:0/1348090230 reap_dead start 2023-06-28T20:07:27.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.392+0000 7efc1e7fc700 1 -- 172.21.15.43:0/1348090230 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc10005b60 con 0x7efc08004850 2023-06-28T20:07:27.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.392+0000 7efc1e7fc700 1 -- 172.21.15.43:0/1348090230 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc10005cc0 con 0x7efc08004850 2023-06-28T20:07:27.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.392+0000 7efc1e7fc700 1 -- 172.21.15.43:0/1348090230 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc100072d0 con 0x7efc08004850 2023-06-28T20:07:27.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.391+0000 7fd1626ef700 1 -- start start 2023-06-28T20:07:27.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.392+0000 7fd1626ef700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd15c158ef0 0x7fd15c06cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.392+0000 7fd1626ef700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd15c159710 0x7fd15c0cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.392+0000 7fd1626ef700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd15c06d610 con 0x7fd15c1592c0 2023-06-28T20:07:27.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.392+0000 7fd1626ef700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd15c15c110 con 0x7fd15c158ef0 2023-06-28T20:07:27.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.392+0000 7fd1626ef700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c15c3f0 con 0x7fd15c159710 2023-06-28T20:07:27.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.391+0000 7fdbd1845700 1 -- start start 2023-06-28T20:07:27.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.392+0000 7fd15bfff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd15c158ef0 0x7fd15c06cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.392+0000 7fd15bfff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd15c158ef0 0x7fd15c06cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:39068/0 (socket says 172.21.15.43:39068) 2023-06-28T20:07:27.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.392+0000 7fd15bfff700 1 -- 172.21.15.43:0/2227856023 learned_addr learned my addr 172.21.15.43:0/2227856023 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.392+0000 7fdbd1845700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbcc06cbb0 0x7fdbcc089500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.392+0000 7fdbd1845700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbcc0ceee0 0x7fdbcc169690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.392+0000 7fdbd1845700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbcc1690f0 0x7fdbcc202730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.392+0000 7fdbd1845700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc089a40 con 0x7fdbcc1690f0 2023-06-28T20:07:27.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.392+0000 7fdbd1845700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc155c00 con 0x7fdbcc0ceee0 2023-06-28T20:07:27.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.392+0000 7fdbd1845700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc0b34a0 con 0x7fdbcc06cbb0 2023-06-28T20:07:27.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.393+0000 7fd160c8c700 1 --2- 172.21.15.43:0/2227856023 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd15c159710 0x7fd15c0cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.393+0000 7fdbcb7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbcc1690f0 0x7fdbcc202730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.393+0000 7fdbcb7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbcc1690f0 0x7fdbcc202730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53014/0 (socket says 172.21.15.43:53014) 2023-06-28T20:07:27.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.393+0000 7fdbcb7fe700 1 -- 172.21.15.43:0/2406547754 learned_addr learned my addr 172.21.15.43:0/2406547754 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.393+0000 7fd15bfff700 1 -- 172.21.15.43:0/2227856023 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd15c159710 msgr2=0x7fd15c0cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.393+0000 7fd15bfff700 1 --2- 172.21.15.43:0/2227856023 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd15c159710 0x7fd15c0cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.393+0000 7fd15bfff700 1 -- 172.21.15.43:0/2227856023 >> v1:172.21.15.178:6789/0 conn(0x7fd15c1592c0 legacy=0x7fd15c0ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.393+0000 7fd15bfff700 1 -- 172.21.15.43:0/2227856023 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd15c15c6d0 con 0x7fd15c158ef0 2023-06-28T20:07:27.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.393+0000 7fd160c8c700 1 --2- 172.21.15.43:0/2227856023 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd15c159710 0x7fd15c0cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:27.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.393+0000 7fdbcb7fe700 1 -- 172.21.15.43:0/2406547754 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbcc06cbb0 msgr2=0x7fdbcc089500 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-28T20:07:27.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.393+0000 7fdbcb7fe700 1 --2- 172.21.15.43:0/2406547754 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdbcc06cbb0 0x7fdbcc089500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.393+0000 7fdbcb7fe700 1 -- 172.21.15.43:0/2406547754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbcc0ceee0 msgr2=0x7fdbcc169690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-28T20:07:27.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.393+0000 7fdbcb7fe700 1 --2- 172.21.15.43:0/2406547754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdbcc0ceee0 0x7fdbcc169690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.393+0000 7fdbcb7fe700 1 -- 172.21.15.43:0/2406547754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc400c6e0 con 0x7fdbcc1690f0 2023-06-28T20:07:27.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.393+0000 7fd15bfff700 1 --2- 172.21.15.43:0/2227856023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd15c158ef0 0x7fd15c06cf10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd14c006fd0 tx=0x7fd14c00d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=646c6977015010a7 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.393+0000 7fdbcb7fe700 1 --2- 172.21.15.43:0/2406547754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbcc1690f0 0x7fdbcc202730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000c370 tx=0x7fdbc000c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.394+0000 7efc25f20700 1 -- 172.21.15.43:0/1348090230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc08004850 msgr2=0x7efc08004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.394+0000 7efc25f20700 1 --2- 172.21.15.43:0/1348090230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc08004850 0x7efc08004cc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efc10001a60 tx=0x7efc08005880 comp rx=0 tx=0).stop 2023-06-28T20:07:27.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.394+0000 7f762883d700 1 Processor -- start 2023-06-28T20:07:27.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.394+0000 7f762883d700 1 -- start start 2023-06-28T20:07:27.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.394+0000 7efc25f20700 1 -- 172.21.15.43:0/1348090230 shutdown_connections 2023-06-28T20:07:27.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.394+0000 7efc25f20700 1 --2- 172.21.15.43:0/1348090230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc08004850 0x7efc08004cc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.394+0000 7efc25f20700 1 -- 172.21.15.43:0/1348090230 >> 172.21.15.43:0/1348090230 conn(0x7efc200d05c0 msgr2=0x7efc200d2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.394+0000 7fd15affd700 1 -- 172.21.15.43:0/2227856023 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c010070 con 0x7fd15c158ef0 2023-06-28T20:07:27.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.395+0000 7efc25f20700 1 -- 172.21.15.43:0/1348090230 shutdown_connections 2023-06-28T20:07:27.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.395+0000 7efc25f20700 1 -- 172.21.15.43:0/1348090230 wait complete. 2023-06-28T20:07:27.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.395+0000 7fd15affd700 1 -- 172.21.15.43:0/2227856023 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14c00da80 con 0x7fd15c158ef0 2023-06-28T20:07:27.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.395+0000 7fd15affd700 1 -- 172.21.15.43:0/2227856023 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14c00de30 con 0x7fd15c158ef0 2023-06-28T20:07:27.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.396+0000 7fd1626ef700 1 -- 172.21.15.43:0/2227856023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd15c158ef0 msgr2=0x7fd15c06cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.396+0000 7fd1626ef700 1 --2- 172.21.15.43:0/2227856023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd15c158ef0 0x7fd15c06cf10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd14c006fd0 tx=0x7fd14c00d330 comp rx=0 tx=0).stop 2023-06-28T20:07:27.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.396+0000 7fcbaaf5a700 1 Processor -- start 2023-06-28T20:07:27.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.396+0000 7fd1626ef700 1 -- 172.21.15.43:0/2227856023 shutdown_connections 2023-06-28T20:07:27.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.396+0000 7fd1626ef700 1 --2- 172.21.15.43:0/2227856023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd15c158ef0 0x7fd15c06cf10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.396+0000 7fd1626ef700 1 --2- 172.21.15.43:0/2227856023 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd15c159710 0x7fd15c0cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.396+0000 7fd1626ef700 1 -- 172.21.15.43:0/2227856023 >> 172.21.15.43:0/2227856023 conn(0x7fd15c069ef0 msgr2=0x7fd15c06a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.395+0000 7f762883d700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f76240cab90 0x7f762406cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.395+0000 7f762883d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7624158c80 0x7f76241590e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.395+0000 7f762883d700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f762415c110 con 0x7f762406a9d0 2023-06-28T20:07:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.395+0000 7f762883d700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76240cd5f0 con 0x7f7624158c80 2023-06-28T20:07:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.395+0000 7f762883d700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f762415c3f0 con 0x7f76240cab90 2023-06-28T20:07:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.396+0000 7fd1626ef700 1 -- 172.21.15.43:0/2227856023 shutdown_connections 2023-06-28T20:07:27.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.396+0000 7fd1626ef700 1 -- 172.21.15.43:0/2227856023 wait complete. 2023-06-28T20:07:27.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.396+0000 7f762259c700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f762406a9d0 0x7f76240ca480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:41328/0 (socket says 172.21.15.43:41328) 2023-06-28T20:07:27.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.396+0000 7f762259c700 1 -- 172.21.15.43:0/2802678014 learned_addr learned my addr 172.21.15.43:0/2802678014 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:27.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.396+0000 7f7621d9b700 1 --2- 172.21.15.43:0/2802678014 >> v2:172.21.15.178:3300/0 conn(0x7f76240cab90 0x7f762406cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.396+0000 7f762159a700 1 -- 172.21.15.43:0/2802678014 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213577740 0 0) 0x7f762415c110 con 0x7f762406a9d0 2023-06-28T20:07:27.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.395+0000 7fdbb3fff700 1 -- 172.21.15.43:0/2406547754 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0005d70 con 0x7fdbcc1690f0 2023-06-28T20:07:27.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.395+0000 7fdbb3fff700 1 -- 172.21.15.43:0/2406547754 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc00070d0 con 0x7fdbcc1690f0 2023-06-28T20:07:27.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.395+0000 7fdbb3fff700 1 -- 172.21.15.43:0/2406547754 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0007440 con 0x7fdbcc1690f0 2023-06-28T20:07:27.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.396+0000 7f762159a700 1 -- 172.21.15.43:0/2802678014 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f760c0035b0 con 0x7f762406a9d0 2023-06-28T20:07:27.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.396+0000 7fdbd1845700 1 -- 172.21.15.43:0/2406547754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbcc0b3780 con 0x7fdbcc1690f0 2023-06-28T20:07:27.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.396+0000 7fdbd1845700 1 -- 172.21.15.43:0/2406547754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbcc0b3d40 con 0x7fdbcc1690f0 2023-06-28T20:07:27.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.396+0000 7efc25f20700 1 Processor -- start 2023-06-28T20:07:27.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.397+0000 7efc25f20700 1 -- start start 2023-06-28T20:07:27.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.397+0000 7f762159a700 1 -- 172.21.15.43:0/2802678014 <== mon.1 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1044243132 0 0) 0x7f760c0035b0 con 0x7f762406a9d0 2023-06-28T20:07:27.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.397+0000 7f762159a700 1 -- 172.21.15.43:0/2802678014 >> v2:172.21.15.178:3300/0 conn(0x7f76240cab90 msgr2=0x7f762406cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.397+0000 7f762159a700 1 --2- 172.21.15.43:0/2802678014 >> v2:172.21.15.178:3300/0 conn(0x7f76240cab90 0x7f762406cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.397+0000 7f762159a700 1 -- 172.21.15.43:0/2802678014 >> v2:172.21.15.43:3300/0 conn(0x7f7624158c80 msgr2=0x7f76241590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.397+0000 7f762159a700 1 --2- 172.21.15.43:0/2802678014 >> v2:172.21.15.43:3300/0 conn(0x7f7624158c80 0x7f76241590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.397+0000 7f762159a700 1 -- 172.21.15.43:0/2802678014 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f762415c6d0 con 0x7f762406a9d0 2023-06-28T20:07:27.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.397+0000 7f762159a700 1 -- 172.21.15.43:0/2802678014 <== mon.1 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f761c001a40 con 0x7f762406a9d0 2023-06-28T20:07:27.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.397+0000 7f762159a700 1 -- 172.21.15.43:0/2802678014 >> v1:172.21.15.178:6789/0 conn(0x7f762406a9d0 legacy=0x7f76240ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.397+0000 7f762159a700 1 --2- 172.21.15.43:0/2802678014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f760c003a00 0x7f760c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.397+0000 7f762159a700 1 --2- 172.21.15.43:0/2802678014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f760c004850 0x7f760c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.397+0000 7f762159a700 1 --2- 172.21.15.43:0/2802678014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f760c005200 0x7f760c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.397+0000 7f762159a700 1 -- 172.21.15.43:0/2802678014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76240cd5f0 con 0x7f760c005200 2023-06-28T20:07:27.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.397+0000 7f762159a700 1 -- 172.21.15.43:0/2802678014 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762415c3f0 con 0x7f760c003a00 2023-06-28T20:07:27.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.397+0000 7f762159a700 1 -- 172.21.15.43:0/2802678014 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f760c00c1f0 con 0x7f760c004850 2023-06-28T20:07:27.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.397+0000 7f79f384f700 1 Processor -- start 2023-06-28T20:07:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.397+0000 7f762259c700 1 --2- 172.21.15.43:0/2802678014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f760c003a00 0x7f760c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.397+0000 7f79f384f700 1 -- start start 2023-06-28T20:07:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.398+0000 7fdbd1845700 1 -- 172.21.15.43:0/2406547754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdbcc06d250 con 0x7fdbcc1690f0 2023-06-28T20:07:27.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.396+0000 7fcbaaf5a700 1 -- start start 2023-06-28T20:07:27.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.398+0000 7fd1626ef700 1 Processor -- start 2023-06-28T20:07:27.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.398+0000 7efc25f20700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc08004850 0x7efc200c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.398+0000 7efc25f20700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc201f2d00 0x7efc200c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.398+0000 7efc25f20700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc200c6f50 0x7efc200cc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.398+0000 7efc25f20700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc200cb260 con 0x7efc201f2d00 2023-06-28T20:07:27.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.398+0000 7efc25f20700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc200cb540 con 0x7efc200c6f50 2023-06-28T20:07:27.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.398+0000 7efc25f20700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc200cb820 con 0x7efc08004850 2023-06-28T20:07:27.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.398+0000 7efc1f7fe700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc08004850 0x7efc200c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7efc1f7fe700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc08004850 0x7efc200c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.43:36998/0 (socket says 172.21.15.43:36998) 2023-06-28T20:07:27.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7efc1f7fe700 1 -- 172.21.15.43:0/3495947151 learned_addr learned my addr 172.21.15.43:0/3495947151 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7efc1ffff700 1 --2- 172.21.15.43:0/3495947151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc200c6f50 0x7efc200cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.398+0000 7f79f384f700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f79ec15c6c0 0x7f79ec0d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7f79f384f700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79ec0b3f70 0x7f79ec0d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7f79f384f700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79ec0b2ad0 con 0x7f79ec0d3bc0 2023-06-28T20:07:27.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7f79f384f700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79ec0b1c30 con 0x7f79ec15c6c0 2023-06-28T20:07:27.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7f79f384f700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec0b1950 con 0x7f79ec0b3f70 2023-06-28T20:07:27.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7efc1f7fe700 1 -- 172.21.15.43:0/3495947151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc200c6f50 msgr2=0x7efc200cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7efc1f7fe700 1 --2- 172.21.15.43:0/3495947151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efc200c6f50 0x7efc200cc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7efc1f7fe700 1 -- 172.21.15.43:0/3495947151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc201f2d00 msgr2=0x7efc200c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7efc1f7fe700 1 --2- 172.21.15.43:0/3495947151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc201f2d00 0x7efc200c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7efc1f7fe700 1 -- 172.21.15.43:0/3495947151 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc200cbb00 con 0x7efc08004850 2023-06-28T20:07:27.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7f79f15eb700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f79ec15c6c0 0x7f79ec0d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7f79f1dec700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79ec0b3f70 0x7f79ec0d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7f79f15eb700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f79ec15c6c0 0x7f79ec0d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:39098/0 (socket says 172.21.15.43:39098) 2023-06-28T20:07:27.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7f79f15eb700 1 -- 172.21.15.43:0/1564902507 learned_addr learned my addr 172.21.15.43:0/1564902507 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7fdbb3fff700 1 -- 172.21.15.43:0/2406547754 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdbc00075a0 con 0x7fdbcc1690f0 2023-06-28T20:07:27.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7fcbaaf5a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcba40cdec0 0x7fcba4160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7fcbaaf5a700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcba4164eb0 0x7fcba406c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7fcbaaf5a700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba41614e0 con 0x7fcba40ce290 2023-06-28T20:07:27.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7fcbaaf5a700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba40ca7b0 con 0x7fcba40cdec0 2023-06-28T20:07:27.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7fcbaaf5a700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba41617c0 con 0x7fcba4164eb0 2023-06-28T20:07:27.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7efc1f7fe700 1 --2- 172.21.15.43:0/3495947151 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc08004850 0x7efc200c5bd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efc1400d810 tx=0x7efc1400db20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.399+0000 7f70f9108700 1 Processor -- start 2023-06-28T20:07:27.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7fcba94f7700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcba4164eb0 0x7fcba406c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7fcba94f7700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcba4164eb0 0x7fcba406c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:39108/0 (socket says 172.21.15.43:39108) 2023-06-28T20:07:27.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7fcba94f7700 1 -- 172.21.15.43:0/2020874616 learned_addr learned my addr 172.21.15.43:0/2020874616 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7fcba94f7700 1 -- 172.21.15.43:0/2020874616 >> v2:172.21.15.43:3300/0 conn(0x7fcba40cdec0 msgr2=0x7fcba4160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7fcba94f7700 1 --2- 172.21.15.43:0/2020874616 >> v2:172.21.15.43:3300/0 conn(0x7fcba40cdec0 0x7fcba4160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7fcba94f7700 1 -- 172.21.15.43:0/2020874616 >> v1:172.21.15.43:6789/0 conn(0x7fcba40ce290 legacy=0x7fcba40ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7fcba94f7700 1 -- 172.21.15.43:0/2020874616 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba4161aa0 con 0x7fcba4164eb0 2023-06-28T20:07:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7f7621d9b700 1 --2- 172.21.15.43:0/2802678014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f760c004850 0x7f760c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7f762259c700 1 -- 172.21.15.43:0/2802678014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f760c004850 msgr2=0x7f760c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7f762259c700 1 --2- 172.21.15.43:0/2802678014 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f760c004850 0x7f760c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7f762259c700 1 -- 172.21.15.43:0/2802678014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f760c005200 msgr2=0x7f760c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7f762259c700 1 --2- 172.21.15.43:0/2802678014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f760c005200 0x7f760c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7f762259c700 1 -- 172.21.15.43:0/2802678014 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760c0035b0 con 0x7f760c003a00 2023-06-28T20:07:27.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7f7622d9d700 1 -- 172.21.15.43:0/2802678014 reap_dead start 2023-06-28T20:07:27.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7f762259c700 1 --2- 172.21.15.43:0/2802678014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f760c003a00 0x7f760c003e10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f761c00a870 tx=0x7f760c005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c762efe1437eb0b server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7f70f9108700 1 -- start start 2023-06-28T20:07:27.439 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7efc07fff700 1 -- 172.21.15.43:0/3495947151 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc14005480 con 0x7efc08004850 2023-06-28T20:07:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7efc07fff700 1 -- 172.21.15.43:0/3495947151 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc14003680 con 0x7efc08004850 2023-06-28T20:07:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7efc07fff700 1 -- 172.21.15.43:0/3495947151 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc14003a30 con 0x7efc08004850 2023-06-28T20:07:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.398+0000 7fd1626ef700 1 -- start start 2023-06-28T20:07:27.440 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7f79f15eb700 1 -- 172.21.15.43:0/1564902507 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79ec0b3f70 msgr2=0x7f79ec0d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7f79f15eb700 1 --2- 172.21.15.43:0/1564902507 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79ec0b3f70 0x7f79ec0d0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7f79f15eb700 1 -- 172.21.15.43:0/1564902507 >> v1:172.21.15.178:6789/0 conn(0x7f79ec0d3bc0 legacy=0x7f79ec0afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7f79f15eb700 1 -- 172.21.15.43:0/1564902507 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec0b2480 con 0x7f79ec15c6c0 2023-06-28T20:07:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7f79f15eb700 1 --2- 172.21.15.43:0/1564902507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79ec15c6c0 0x7f79ec0d0ab0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f79dc006fd0 tx=0x7f79dc009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed819ebcfbb65f64 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7fdbb3fff700 1 --2- 172.21.15.43:0/2406547754 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fdbb806ad30 0x7fdbb806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7fdbcaffd700 1 --2- 172.21.15.43:0/2406547754 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fdbb806ad30 0x7fdbb806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.400+0000 7fdbb3fff700 1 -- 172.21.15.43:0/2406547754 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdbc005b4d0 con 0x7fdbcc1690f0 2023-06-28T20:07:27.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7fdbb3fff700 1 --2- 172.21.15.43:0/2406547754 >> [v2:172.21.15.178:6816/3953458472,v1:172.21.15.178:6817/3953458472] conn(0x7fdbb8073560 0x7fdbb8075970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7fdbb3fff700 1 -- 172.21.15.43:0/2406547754 --> [v2:172.21.15.178:6816/3953458472,v1:172.21.15.178:6817/3953458472] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdbb8076020 con 0x7fdbb8073560 2023-06-28T20:07:27.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7fdbb3fff700 1 -- 172.21.15.43:0/2406547754 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbc0053900 con 0x7fdbcc1690f0 2023-06-28T20:07:27.442 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7fdbcaffd700 1 --2- 172.21.15.43:0/2406547754 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fdbb806ad30 0x7fdbb806d1e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4000f80 tx=0x7fdbc4009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7f70f9108700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70f415cae0 0x7f70f40cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7f70f9108700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f70f40cfe00 0x7f70f406cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7f70f9108700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f40d0680 con 0x7f70f406d310 2023-06-28T20:07:27.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7f70f9108700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f406bb20 con 0x7f70f40cfe00 2023-06-28T20:07:27.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7f70f9108700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f406be00 con 0x7f70f415cae0 2023-06-28T20:07:27.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7fdbca7fc700 1 --2- 172.21.15.43:0/2406547754 >> [v2:172.21.15.178:6816/3953458472,v1:172.21.15.178:6817/3953458472] conn(0x7fdbb8073560 0x7fdbb8075970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7fd1626ef700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd15c158ef0 0x7fd15c1517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7fd1626ef700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd15c1592c0 0x7fd15c153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7fd1626ef700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd15c159710 0x7fd15c0b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7fd1626ef700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c1f6610 con 0x7fd15c158ef0 2023-06-28T20:07:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7fd1626ef700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c151d30 con 0x7fd15c1592c0 2023-06-28T20:07:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7fd1626ef700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15c154ac0 con 0x7fd15c159710 2023-06-28T20:07:27.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7fd15b7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd15c1592c0 0x7fd15c153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7fd15b7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd15c1592c0 0x7fd15c153cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:39116/0 (socket says 172.21.15.43:39116) 2023-06-28T20:07:27.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7fd15b7fe700 1 -- 172.21.15.43:0/3809256105 learned_addr learned my addr 172.21.15.43:0/3809256105 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7fd160c8c700 1 --2- 172.21.15.43:0/3809256105 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd15c159710 0x7fd15c0b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7f70f259c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70f415cae0 0x7f70f40cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7f70f259c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70f415cae0 0x7f70f40cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.43:37030/0 (socket says 172.21.15.43:37030) 2023-06-28T20:07:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7f70f259c700 1 -- 172.21.15.43:0/927967440 learned_addr learned my addr 172.21.15.43:0/927967440 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7fcba94f7700 1 --2- 172.21.15.43:0/2020874616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcba4164eb0 0x7fcba406c6e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcba40cfab0 tx=0x7fcb9c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b3807e4aa3e6f2 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7fcba37fe700 1 -- 172.21.15.43:0/2020874616 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c00e070 con 0x7fcba4164eb0 2023-06-28T20:07:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7fcba37fe700 1 -- 172.21.15.43:0/2020874616 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9c003a20 con 0x7fcba4164eb0 2023-06-28T20:07:27.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7fcba37fe700 1 -- 172.21.15.43:0/2020874616 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c003dd0 con 0x7fcba4164eb0 2023-06-28T20:07:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7f762159a700 1 -- 172.21.15.43:0/2802678014 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c012600 con 0x7f760c003a00 2023-06-28T20:07:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7f762159a700 1 -- 172.21.15.43:0/2802678014 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761c012780 con 0x7f760c003a00 2023-06-28T20:07:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7f762159a700 1 -- 172.21.15.43:0/2802678014 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761c012b30 con 0x7f760c003a00 2023-06-28T20:07:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7f762883d700 1 -- 172.21.15.43:0/2802678014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f760c003a00 msgr2=0x7f760c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7f762883d700 1 --2- 172.21.15.43:0/2802678014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f760c003a00 0x7f760c003e10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f761c00a870 tx=0x7f760c005880 comp rx=0 tx=0).stop 2023-06-28T20:07:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7f762883d700 1 -- 172.21.15.43:0/2802678014 shutdown_connections 2023-06-28T20:07:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7f762883d700 1 --2- 172.21.15.43:0/2802678014 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f760c003a00 0x7f760c003e10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7f762883d700 1 -- 172.21.15.43:0/2802678014 >> 172.21.15.43:0/2802678014 conn(0x7f7624069ef0 msgr2=0x7f762406a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7efc25f20700 1 -- 172.21.15.43:0/3495947151 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc200c6110 con 0x7efc08004850 2023-06-28T20:07:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7efc25f20700 1 -- 172.21.15.43:0/3495947151 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc201f26a0 con 0x7efc08004850 2023-06-28T20:07:27.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7fd15b7fe700 1 -- 172.21.15.43:0/3809256105 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd15c159710 msgr2=0x7fd15c0b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7fd15b7fe700 1 --2- 172.21.15.43:0/3809256105 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd15c159710 0x7fd15c0b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7fd15b7fe700 1 -- 172.21.15.43:0/3809256105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd15c158ef0 msgr2=0x7fd15c1517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7fd15b7fe700 1 --2- 172.21.15.43:0/3809256105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd15c158ef0 0x7fd15c1517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7fd15b7fe700 1 -- 172.21.15.43:0/3809256105 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c008580 con 0x7fd15c1592c0 2023-06-28T20:07:27.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.401+0000 7f79e3fff700 1 -- 172.21.15.43:0/1564902507 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc005e30 con 0x7f79ec15c6c0 2023-06-28T20:07:27.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7f79e3fff700 1 -- 172.21.15.43:0/1564902507 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79dc004030 con 0x7f79ec15c6c0 2023-06-28T20:07:27.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7f79e3fff700 1 -- 172.21.15.43:0/1564902507 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc0043e0 con 0x7f79ec15c6c0 2023-06-28T20:07:27.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7f70f259c700 1 -- 172.21.15.43:0/927967440 >> v2:172.21.15.43:3300/0 conn(0x7f70f40cfe00 msgr2=0x7f70f406cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7f70f259c700 1 --2- 172.21.15.43:0/927967440 >> v2:172.21.15.43:3300/0 conn(0x7f70f40cfe00 0x7f70f406cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7f70f259c700 1 -- 172.21.15.43:0/927967440 >> v1:172.21.15.43:6789/0 conn(0x7f70f406d310 legacy=0x7f70f406d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7f70f259c700 1 -- 172.21.15.43:0/927967440 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f406c0e0 con 0x7f70f415cae0 2023-06-28T20:07:27.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7fd15b7fe700 1 --2- 172.21.15.43:0/3809256105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd15c1592c0 0x7fd15c153cf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd15000c4e0 tx=0x7fd15000c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7f762883d700 1 -- 172.21.15.43:0/2802678014 shutdown_connections 2023-06-28T20:07:27.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7f762883d700 1 -- 172.21.15.43:0/2802678014 wait complete. 2023-06-28T20:07:27.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7fdbca7fc700 1 --2- 172.21.15.43:0/2406547754 >> [v2:172.21.15.178:6816/3953458472,v1:172.21.15.178:6817/3953458472] conn(0x7fdbb8073560 0x7fdbb8075970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.403+0000 7fcbaaf5a700 1 -- 172.21.15.43:0/2020874616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcba4164eb0 msgr2=0x7fcba406c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.403+0000 7fcbaaf5a700 1 --2- 172.21.15.43:0/2020874616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcba4164eb0 0x7fcba406c6e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcba40cfab0 tx=0x7fcb9c00b330 comp rx=0 tx=0).stop 2023-06-28T20:07:27.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.403+0000 7f70f259c700 1 --2- 172.21.15.43:0/927967440 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70f415cae0 0x7f70f40cf7e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009e50 tx=0x7f70ec00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94021d37dd927d8d server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.403+0000 7f70f1d9b700 1 -- 172.21.15.43:0/927967440 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec00e070 con 0x7f70f415cae0 2023-06-28T20:07:27.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.403+0000 7f70f1d9b700 1 -- 172.21.15.43:0/927967440 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ec003a20 con 0x7f70f415cae0 2023-06-28T20:07:27.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.403+0000 7f70f1d9b700 1 -- 172.21.15.43:0/927967440 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec003dd0 con 0x7f70f415cae0 2023-06-28T20:07:27.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7fd158ff9700 1 -- 172.21.15.43:0/3809256105 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1500052d0 con 0x7fd15c1592c0 2023-06-28T20:07:27.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7fd158ff9700 1 -- 172.21.15.43:0/3809256105 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd150005430 con 0x7fd15c1592c0 2023-06-28T20:07:27.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7fd158ff9700 1 -- 172.21.15.43:0/3809256105 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1500038a0 con 0x7fd15c1592c0 2023-06-28T20:07:27.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.402+0000 7fd1626ef700 1 -- 172.21.15.43:0/3809256105 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15c1547e0 con 0x7fd15c1592c0 2023-06-28T20:07:27.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.403+0000 7fd1626ef700 1 -- 172.21.15.43:0/3809256105 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd15c0b1d20 con 0x7fd15c1592c0 2023-06-28T20:07:27.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.403+0000 7f79f384f700 1 -- 172.21.15.43:0/1564902507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79ec15c6c0 msgr2=0x7f79ec0d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.403+0000 7f79f384f700 1 --2- 172.21.15.43:0/1564902507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79ec15c6c0 0x7f79ec0d0ab0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f79dc006fd0 tx=0x7f79dc009bd0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.404+0000 7fdbb3fff700 1 -- 172.21.15.43:0/2406547754 <== osd.7 v2:172.21.15.178:6816/3953458472 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fdbb8076020 con 0x7fdbb8073560 2023-06-28T20:07:27.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.404+0000 7fcbaaf5a700 1 -- 172.21.15.43:0/2020874616 shutdown_connections 2023-06-28T20:07:27.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.404+0000 7fcbaaf5a700 1 --2- 172.21.15.43:0/2020874616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcba4164eb0 0x7fcba406c6e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.404+0000 7fcbaaf5a700 1 --2- 172.21.15.43:0/2020874616 >> v2:172.21.15.43:3300/0 conn(0x7fcba40cdec0 0x7fcba4160eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.404+0000 7fcbaaf5a700 1 -- 172.21.15.43:0/2020874616 >> 172.21.15.43:0/2020874616 conn(0x7fcba4069ef0 msgr2=0x7fcba406a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.404+0000 7f79f384f700 1 -- 172.21.15.43:0/1564902507 shutdown_connections 2023-06-28T20:07:27.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.404+0000 7f79f384f700 1 --2- 172.21.15.43:0/1564902507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79ec15c6c0 0x7f79ec0d0ab0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.404+0000 7f79f384f700 1 --2- 172.21.15.43:0/1564902507 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79ec0b3f70 0x7f79ec0d0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.404+0000 7f79f384f700 1 -- 172.21.15.43:0/1564902507 >> 172.21.15.43:0/1564902507 conn(0x7f79ec09c210 msgr2=0x7f79ec0d2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.404+0000 7fcbaaf5a700 1 -- 172.21.15.43:0/2020874616 shutdown_connections 2023-06-28T20:07:27.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.404+0000 7efc25f20700 1 -- 172.21.15.43:0/3495947151 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efc200d1a40 con 0x7efc08004850 2023-06-28T20:07:27.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.404+0000 7efc07fff700 1 -- 172.21.15.43:0/3495947151 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efc14003b90 con 0x7efc08004850 2023-06-28T20:07:27.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.404+0000 7f762883d700 1 Processor -- start 2023-06-28T20:07:27.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.404+0000 7fd1626ef700 1 -- 172.21.15.43:0/3809256105 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd13c000fc0 con 0x7fd15c1592c0 2023-06-28T20:07:27.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.404+0000 7f762883d700 1 -- start start 2023-06-28T20:07:27.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.405+0000 7efc07fff700 1 --2- 172.21.15.43:0/3495947151 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efc0c06adb0 0x7efc0c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.405+0000 7f70f9108700 1 -- 172.21.15.43:0/927967440 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70f415cae0 msgr2=0x7f70f40cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.405+0000 7f70f9108700 1 --2- 172.21.15.43:0/927967440 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70f415cae0 0x7f70f40cf7e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009e50 tx=0x7f70ec00b330 comp rx=0 tx=0).stop 2023-06-28T20:07:27.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.405+0000 7efc07fff700 1 -- 172.21.15.43:0/3495947151 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efc140892c0 con 0x7efc08004850 2023-06-28T20:07:27.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.405+0000 7f70f9108700 1 -- 172.21.15.43:0/927967440 shutdown_connections 2023-06-28T20:07:27.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.405+0000 7f70f9108700 1 --2- 172.21.15.43:0/927967440 >> v2:172.21.15.43:3300/0 conn(0x7f70f40cfe00 0x7f70f406cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.405+0000 7f70f9108700 1 --2- 172.21.15.43:0/927967440 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70f415cae0 0x7f70f40cf7e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.405+0000 7f70f9108700 1 -- 172.21.15.43:0/927967440 >> 172.21.15.43:0/927967440 conn(0x7f70f40664b0 msgr2=0x7f70f40d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.405+0000 7efc07fff700 1 --2- 172.21.15.43:0/3495947151 >> [v2:172.21.15.43:6816/2901255345,v1:172.21.15.43:6817/2901255345] conn(0x7efc0c0735e0 0x7efc0c0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.405+0000 7efc07fff700 1 -- 172.21.15.43:0/3495947151 --> [v2:172.21.15.43:6816/2901255345,v1:172.21.15.43:6817/2901255345] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efc0c0760a0 con 0x7efc0c0735e0 2023-06-28T20:07:27.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.405+0000 7f79f384f700 1 -- 172.21.15.43:0/1564902507 shutdown_connections 2023-06-28T20:07:27.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.405+0000 7f70f9108700 1 -- 172.21.15.43:0/927967440 shutdown_connections 2023-06-28T20:07:27.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.405+0000 7f79f384f700 1 -- 172.21.15.43:0/1564902507 wait complete. 2023-06-28T20:07:27.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.405+0000 7f762883d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f760c003a00 0x7f762406f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.406+0000 7f762883d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76241ed690 0x7f76240b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.406+0000 7f762883d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7624159ee0 0x7f762415a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.406+0000 7f762883d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76241f2360 con 0x7f76241ed690 2023-06-28T20:07:27.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.406+0000 7f762883d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f762415a8c0 con 0x7f760c003a00 2023-06-28T20:07:27.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.406+0000 7f762883d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76241f16e0 con 0x7f7624159ee0 2023-06-28T20:07:27.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.404+0000 7fcbaaf5a700 1 -- 172.21.15.43:0/2020874616 wait complete. 2023-06-28T20:07:27.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.406+0000 7f7621d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76241ed690 0x7f76240b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.406+0000 7f7621d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76241ed690 0x7f76240b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53068/0 (socket says 172.21.15.43:53068) 2023-06-28T20:07:27.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.406+0000 7f7621d9b700 1 -- 172.21.15.43:0/2574685447 learned_addr learned my addr 172.21.15.43:0/2574685447 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.406+0000 7fcbaaf5a700 1 Processor -- start 2023-06-28T20:07:27.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.406+0000 7f7621d9b700 1 -- 172.21.15.43:0/2574685447 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7624159ee0 msgr2=0x7f762415a380 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-28T20:07:27.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.406+0000 7f7621d9b700 1 --2- 172.21.15.43:0/2574685447 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7624159ee0 0x7f762415a380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.406+0000 7f7621d9b700 1 -- 172.21.15.43:0/2574685447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f760c003a00 msgr2=0x7f762406f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.406+0000 7f7621d9b700 1 --2- 172.21.15.43:0/2574685447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f760c003a00 0x7f762406f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.406+0000 7f7621d9b700 1 -- 172.21.15.43:0/2574685447 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76241f19c0 con 0x7f76241ed690 2023-06-28T20:07:27.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.406+0000 7f7621d9b700 1 --2- 172.21.15.43:0/2574685447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76241ed690 0x7f76240b41c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7618011630 tx=0x7f76180119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.407+0000 7efc1ffff700 1 --2- 172.21.15.43:0/3495947151 >> [v2:172.21.15.43:6816/2901255345,v1:172.21.15.43:6817/2901255345] conn(0x7efc0c0735e0 0x7efc0c0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.405+0000 7f70f9108700 1 -- 172.21.15.43:0/927967440 wait complete. 2023-06-28T20:07:27.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.407+0000 7efc1ffff700 1 --2- 172.21.15.43:0/3495947151 >> [v2:172.21.15.43:6816/2901255345,v1:172.21.15.43:6817/2901255345] conn(0x7efc0c0735e0 0x7efc0c0759f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.407+0000 7f79f384f700 1 Processor -- start 2023-06-28T20:07:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.407+0000 7f70f9108700 1 Processor -- start 2023-06-28T20:07:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.408+0000 7efc1effd700 1 --2- 172.21.15.43:0/3495947151 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efc0c06adb0 0x7efc0c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.406+0000 7fcbaaf5a700 1 -- start start 2023-06-28T20:07:27.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.407+0000 7fcbaaf5a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcba40cdec0 0x7fcba40b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.407+0000 7fcbaaf5a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba40ce290 0x7fcba41ff090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.407+0000 7fcbaaf5a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcba4164eb0 0x7fcba4158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.407+0000 7fcbaaf5a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba42004c0 con 0x7fcba40ce290 2023-06-28T20:07:27.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.407+0000 7fcbaaf5a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba41ff5d0 con 0x7fcba4164eb0 2023-06-28T20:07:27.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.407+0000 7fcbaaf5a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba4201b90 con 0x7fcba40cdec0 2023-06-28T20:07:27.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.408+0000 7fcba3fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba40ce290 0x7fcba41ff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.408+0000 7fcba3fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba40ce290 0x7fcba41ff090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53074/0 (socket says 172.21.15.43:53074) 2023-06-28T20:07:27.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.408+0000 7fcba3fff700 1 -- 172.21.15.43:0/1680171515 learned_addr learned my addr 172.21.15.43:0/1680171515 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.407+0000 7f79f384f700 1 -- start start 2023-06-28T20:07:27.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.408+0000 7efc1effd700 1 --2- 172.21.15.43:0/3495947151 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efc0c06adb0 0x7efc0c06d260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efc10001eb0 tx=0x7efc1000f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.408+0000 7efc07fff700 1 -- 172.21.15.43:0/3495947151 <== osd.3 v2:172.21.15.43:6816/2901255345 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7efc0c0760a0 con 0x7efc0c0735e0 2023-06-28T20:07:27.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7fcba3fff700 1 -- 172.21.15.43:0/1680171515 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcba40cdec0 msgr2=0x7fcba40b4560 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-28T20:07:27.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7fcba3fff700 1 --2- 172.21.15.43:0/1680171515 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcba40cdec0 0x7fcba40b4560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7fcba3fff700 1 -- 172.21.15.43:0/1680171515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcba4164eb0 msgr2=0x7fcba4158370 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-28T20:07:27.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7fcba3fff700 1 --2- 172.21.15.43:0/1680171515 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcba4164eb0 0x7fcba4158370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7fcba3fff700 1 -- 172.21.15.43:0/1680171515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb9c009b30 con 0x7fcba40ce290 2023-06-28T20:07:27.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f79f384f700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79ec0b3f70 0x7f79ec1f5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f79f384f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79ec0d3bc0 0x7f79ec0b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f79f384f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79ec15c6c0 0x7f79ec0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f79f384f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec1fa2a0 con 0x7f79ec0d3bc0 2023-06-28T20:07:27.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f79f384f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec154300 con 0x7f79ec15c6c0 2023-06-28T20:07:27.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f79f384f700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec1e6700 con 0x7f79ec0b3f70 2023-06-28T20:07:27.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f79f0dea700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79ec0d3bc0 0x7f79ec0b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.408+0000 7f70f9108700 1 -- start start 2023-06-28T20:07:27.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f70f9108700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70f406d310 0x7f70f40b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f70f9108700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f40cfe00 0x7f70f40c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f70f9108700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70f415cae0 0x7f70f40b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f70f9108700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f40b5930 con 0x7f70f40cfe00 2023-06-28T20:07:27.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f70f9108700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f40b4d20 con 0x7f70f415cae0 2023-06-28T20:07:27.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f70f9108700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f40b5000 con 0x7f70f406d310 2023-06-28T20:07:27.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7fcba3fff700 1 --2- 172.21.15.43:0/1680171515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba40ce290 0x7fcba41ff090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000c4d0 tx=0x7fcb9000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f79f0dea700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79ec0d3bc0 0x7f79ec0b49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53080/0 (socket says 172.21.15.43:53080) 2023-06-28T20:07:27.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f79f0dea700 1 -- 172.21.15.43:0/440453923 learned_addr learned my addr 172.21.15.43:0/440453923 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f79f1dec700 1 --2- 172.21.15.43:0/440453923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79ec15c6c0 0x7f79ec0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f79f15eb700 1 --2- 172.21.15.43:0/440453923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79ec0b3f70 0x7f79ec1f5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f70f259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f40cfe00 0x7f70f40c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f70f259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f40cfe00 0x7f70f40c8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53084/0 (socket says 172.21.15.43:53084) 2023-06-28T20:07:27.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f70f259c700 1 -- 172.21.15.43:0/4196329587 learned_addr learned my addr 172.21.15.43:0/4196329587 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f79f15eb700 1 -- 172.21.15.43:0/440453923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79ec15c6c0 msgr2=0x7f79ec0b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f79f15eb700 1 --2- 172.21.15.43:0/440453923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79ec15c6c0 0x7f79ec0b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f79f15eb700 1 -- 172.21.15.43:0/440453923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79ec0d3bc0 msgr2=0x7f79ec0b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f79f15eb700 1 --2- 172.21.15.43:0/440453923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79ec0d3bc0 0x7f79ec0b49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.409+0000 7f79f15eb700 1 -- 172.21.15.43:0/440453923 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc009500 con 0x7f79ec0b3f70 2023-06-28T20:07:27.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.410+0000 7f70f259c700 1 -- 172.21.15.43:0/4196329587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70f406d310 msgr2=0x7f70f40b5dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-28T20:07:27.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.410+0000 7f70f259c700 1 --2- 172.21.15.43:0/4196329587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70f406d310 0x7f70f40b5dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.410+0000 7f70f259c700 1 -- 172.21.15.43:0/4196329587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70f415cae0 msgr2=0x7f70f40b0ee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-28T20:07:27.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.410+0000 7f70f259c700 1 --2- 172.21.15.43:0/4196329587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70f415cae0 0x7f70f40b0ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.410+0000 7f70f259c700 1 -- 172.21.15.43:0/4196329587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec009b30 con 0x7f70f40cfe00 2023-06-28T20:07:27.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.410+0000 7f79f15eb700 1 --2- 172.21.15.43:0/440453923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79ec0b3f70 0x7f79ec1f5ad0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f79ec06a130 tx=0x7f79dc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.410+0000 7f70f259c700 1 --2- 172.21.15.43:0/4196329587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f40cfe00 0x7f70f40c8f50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f70f406bf40 tx=0x7f70ec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.410+0000 7fd158ff9700 1 -- 172.21.15.43:0/3809256105 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd150003a00 con 0x7fd15c1592c0 2023-06-28T20:07:27.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.410+0000 7f79e1ffb700 1 -- 172.21.15.43:0/440453923 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc003cb0 con 0x7f79ec0b3f70 2023-06-28T20:07:27.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.410+0000 7f79e1ffb700 1 -- 172.21.15.43:0/440453923 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79dc003e30 con 0x7f79ec0b3f70 2023-06-28T20:07:27.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.410+0000 7f79e1ffb700 1 -- 172.21.15.43:0/440453923 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc01c660 con 0x7f79ec0b3f70 2023-06-28T20:07:27.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.411+0000 7f79f384f700 1 -- 172.21.15.43:0/440453923 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ec1fa6c0 con 0x7f79ec0b3f70 2023-06-28T20:07:27.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.411+0000 7fd158ff9700 1 --2- 172.21.15.43:0/3809256105 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd14806ad30 0x7fd14806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.411+0000 7fd158ff9700 1 -- 172.21.15.43:0/3809256105 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd150088b80 con 0x7fd15c1592c0 2023-06-28T20:07:27.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.411+0000 7fd158ff9700 1 --2- 172.21.15.43:0/3809256105 >> [v2:172.21.15.178:6820/2918661202,v1:172.21.15.178:6822/2918661202] conn(0x7fd148073560 0x7fd148075970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.411+0000 7fd158ff9700 1 -- 172.21.15.43:0/3809256105 --> [v2:172.21.15.178:6820/2918661202,v1:172.21.15.178:6822/2918661202] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd148076020 con 0x7fd148073560 2023-06-28T20:07:27.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.411+0000 7f79f384f700 1 -- 172.21.15.43:0/440453923 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ec1f9be0 con 0x7f79ec0b3f70 2023-06-28T20:07:27.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.412+0000 7f79f384f700 1 -- 172.21.15.43:0/440453923 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f79ec0b2760 con 0x7f79ec0b3f70 2023-06-28T20:07:27.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.412+0000 7f79e1ffb700 1 -- 172.21.15.43:0/440453923 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f79dc01c7c0 con 0x7f79ec0b3f70 2023-06-28T20:07:27.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.412+0000 7fd160c8c700 1 --2- 172.21.15.43:0/3809256105 >> [v2:172.21.15.178:6820/2918661202,v1:172.21.15.178:6822/2918661202] conn(0x7fd148073560 0x7fd148075970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.412+0000 7fd160c8c700 1 --2- 172.21.15.43:0/3809256105 >> [v2:172.21.15.178:6820/2918661202,v1:172.21.15.178:6822/2918661202] conn(0x7fd148073560 0x7fd148075970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.413+0000 7f79e1ffb700 1 --2- 172.21.15.43:0/440453923 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f79d006ace0 0x7f79d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.413+0000 7fd158ff9700 1 -- 172.21.15.43:0/3809256105 <== osd.4 v2:172.21.15.178:6820/2918661202 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd148076020 con 0x7fd148073560 2023-06-28T20:07:27.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.413+0000 7f79f0dea700 1 --2- 172.21.15.43:0/440453923 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f79d006ace0 0x7f79d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.413+0000 7f79e1ffb700 1 -- 172.21.15.43:0/440453923 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f79dc08a3c0 con 0x7f79ec0b3f70 2023-06-28T20:07:27.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.413+0000 7f79e1ffb700 1 --2- 172.21.15.43:0/440453923 >> [v2:172.21.15.178:6800/2113985646,v1:172.21.15.178:6801/2113985646] conn(0x7f79d0073490 0x7f79d00758a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.413+0000 7f79e1ffb700 1 -- 172.21.15.43:0/440453923 --> [v2:172.21.15.178:6800/2113985646,v1:172.21.15.178:6801/2113985646] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f79d0075f50 con 0x7f79d0073490 2023-06-28T20:07:27.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.413+0000 7f79f0dea700 1 --2- 172.21.15.43:0/440453923 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f79d006ace0 0x7f79d006d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0b56e0 tx=0x7f79e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.414+0000 7f79f1dec700 1 --2- 172.21.15.43:0/440453923 >> [v2:172.21.15.178:6800/2113985646,v1:172.21.15.178:6801/2113985646] conn(0x7f79d0073490 0x7f79d00758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.414+0000 7f79f1dec700 1 --2- 172.21.15.43:0/440453923 >> [v2:172.21.15.178:6800/2113985646,v1:172.21.15.178:6801/2113985646] conn(0x7f79d0073490 0x7f79d00758a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.414+0000 7f7612ffd700 1 -- 172.21.15.43:0/2574685447 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761800c070 con 0x7f76241ed690 2023-06-28T20:07:27.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.415+0000 7f70db7fe700 1 -- 172.21.15.43:0/4196329587 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec004440 con 0x7f70f40cfe00 2023-06-28T20:07:27.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.415+0000 7f70db7fe700 1 -- 172.21.15.43:0/4196329587 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ec0045a0 con 0x7f70f40cfe00 2023-06-28T20:07:27.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.415+0000 7f70db7fe700 1 -- 172.21.15.43:0/4196329587 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec004930 con 0x7f70f40cfe00 2023-06-28T20:07:27.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.414+0000 7fd15bfff700 1 --2- 172.21.15.43:0/3809256105 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd14806ad30 0x7fd14806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.415+0000 7f762883d700 1 -- 172.21.15.43:0/2574685447 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76241f1ca0 con 0x7f76241ed690 2023-06-28T20:07:27.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.415+0000 7f762883d700 1 -- 172.21.15.43:0/2574685447 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76240b4700 con 0x7f76241ed690 2023-06-28T20:07:27.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.416+0000 7f7612ffd700 1 -- 172.21.15.43:0/2574685447 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7618015930 con 0x7f76241ed690 2023-06-28T20:07:27.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.416+0000 7f7612ffd700 1 -- 172.21.15.43:0/2574685447 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618015cc0 con 0x7f76241ed690 2023-06-28T20:07:27.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.415+0000 7f70f9108700 1 -- 172.21.15.43:0/4196329587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f40b52e0 con 0x7f70f40cfe00 2023-06-28T20:07:27.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.415+0000 7f70f9108700 1 -- 172.21.15.43:0/4196329587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f40c9490 con 0x7f70f40cfe00 2023-06-28T20:07:27.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.415+0000 7efc07fff700 1 -- 172.21.15.43:0/3495947151 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc140543e0 con 0x7efc08004850 2023-06-28T20:07:27.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.415+0000 7fd15bfff700 1 --2- 172.21.15.43:0/3809256105 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd14806ad30 0x7fd14806d1e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd14c00d370 tx=0x7fd14c014000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.416+0000 7f79e1ffb700 1 -- 172.21.15.43:0/440453923 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79dc055460 con 0x7f79ec0b3f70 2023-06-28T20:07:27.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.416+0000 7f79e1ffb700 1 -- 172.21.15.43:0/440453923 <== osd.6 v2:172.21.15.178:6800/2113985646 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f79d0075f50 con 0x7f79d0073490 2023-06-28T20:07:27.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.417+0000 7f762883d700 1 -- 172.21.15.43:0/2574685447 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7608000fc0 con 0x7f76241ed690 2023-06-28T20:07:27.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.417+0000 7fcba17fa700 1 -- 172.21.15.43:0/1680171515 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb900054e0 con 0x7fcba40ce290 2023-06-28T20:07:27.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.417+0000 7fcba17fa700 1 -- 172.21.15.43:0/1680171515 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb90003680 con 0x7fcba40ce290 2023-06-28T20:07:27.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.417+0000 7fcba17fa700 1 -- 172.21.15.43:0/1680171515 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90003a30 con 0x7fcba40ce290 2023-06-28T20:07:27.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.417+0000 7fcbaaf5a700 1 -- 172.21.15.43:0/1680171515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba41ff840 con 0x7fcba40ce290 2023-06-28T20:07:27.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.417+0000 7fcbaaf5a700 1 -- 172.21.15.43:0/1680171515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba41ffe00 con 0x7fcba40ce290 2023-06-28T20:07:27.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.417+0000 7f7612ffd700 1 -- 172.21.15.43:0/2574685447 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7618013400 con 0x7f76241ed690 2023-06-28T20:07:27.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.418+0000 7f7612ffd700 1 --2- 172.21.15.43:0/2574685447 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f760406ace0 0x7f760406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.418+0000 7f7612ffd700 1 -- 172.21.15.43:0/2574685447 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f76180895b0 con 0x7f76241ed690 2023-06-28T20:07:27.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.418+0000 7f7612ffd700 1 --2- 172.21.15.43:0/2574685447 >> [v2:172.21.15.43:6808/894763335,v1:172.21.15.43:6809/894763335] conn(0x7f7604073510 0x7f7604075920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.418+0000 7f7612ffd700 1 -- 172.21.15.43:0/2574685447 --> [v2:172.21.15.43:6808/894763335,v1:172.21.15.43:6809/894763335] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7604075fd0 con 0x7f7604073510 2023-06-28T20:07:27.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.418+0000 7f70db7fe700 1 -- 172.21.15.43:0/4196329587 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f70ec012430 con 0x7f70f40cfe00 2023-06-28T20:07:27.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.418+0000 7f70db7fe700 1 --2- 172.21.15.43:0/4196329587 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f70e006ace0 0x7f70e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.418+0000 7f70db7fe700 1 -- 172.21.15.43:0/4196329587 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f70ec08ac00 con 0x7f70f40cfe00 2023-06-28T20:07:27.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.419+0000 7f70f9108700 1 --2- 172.21.15.43:0/4196329587 >> [v2:172.21.15.43:6824/1523179953,v1:172.21.15.43:6825/1523179953] conn(0x7f70d40015e0 0x7f70d4003a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.419+0000 7f70f9108700 1 -- 172.21.15.43:0/4196329587 --> [v2:172.21.15.43:6824/1523179953,v1:172.21.15.43:6825/1523179953] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f70d4006bc0 con 0x7f70d40015e0 2023-06-28T20:07:27.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.418+0000 7f7622d9d700 1 --2- 172.21.15.43:0/2574685447 >> [v2:172.21.15.43:6808/894763335,v1:172.21.15.43:6809/894763335] conn(0x7f7604073510 0x7f7604075920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.419+0000 7f762259c700 1 --2- 172.21.15.43:0/2574685447 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f760406ace0 0x7f760406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.419+0000 7f7622d9d700 1 --2- 172.21.15.43:0/2574685447 >> [v2:172.21.15.43:6808/894763335,v1:172.21.15.43:6809/894763335] conn(0x7f7604073510 0x7f7604075920 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.419+0000 7f70f2d9d700 1 --2- 172.21.15.43:0/4196329587 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f70e006ace0 0x7f70e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.419+0000 7f762259c700 1 --2- 172.21.15.43:0/2574685447 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f760406ace0 0x7f760406d190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f76240b2f70 tx=0x7f761c00af90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.420+0000 7f70f2d9d700 1 --2- 172.21.15.43:0/4196329587 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f70e006ace0 0x7f70e006d190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f70e4001580 tx=0x7f70e4007a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.420+0000 7f7612ffd700 1 -- 172.21.15.43:0/2574685447 <== osd.1 v2:172.21.15.43:6808/894763335 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7604075fd0 con 0x7f7604073510 2023-06-28T20:07:27.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.421+0000 7f7612ffd700 1 -- 172.21.15.43:0/2574685447 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7618054650 con 0x7f76241ed690 2023-06-28T20:07:27.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.421+0000 7f70f359e700 1 --2- 172.21.15.43:0/4196329587 >> [v2:172.21.15.43:6824/1523179953,v1:172.21.15.43:6825/1523179953] conn(0x7f70d40015e0 0x7f70d4003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.421+0000 7fcba17fa700 1 -- 172.21.15.43:0/1680171515 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcb90003b90 con 0x7fcba40ce290 2023-06-28T20:07:27.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.421+0000 7f70f359e700 1 --2- 172.21.15.43:0/4196329587 >> [v2:172.21.15.43:6824/1523179953,v1:172.21.15.43:6825/1523179953] conn(0x7f70d40015e0 0x7f70d4003a90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.421+0000 7fcba17fa700 1 --2- 172.21.15.43:0/1680171515 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fcb8806adb0 0x7fcb8806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.422+0000 7fcba17fa700 1 -- 172.21.15.43:0/1680171515 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcb900887f0 con 0x7fcba40ce290 2023-06-28T20:07:27.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.422+0000 7fd158ff9700 1 -- 172.21.15.43:0/3809256105 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd150053ca0 con 0x7fd15c1592c0 2023-06-28T20:07:27.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.422+0000 7f70db7fe700 1 -- 172.21.15.43:0/4196329587 <== osd.2 v2:172.21.15.43:6824/1523179953 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f70d4006bc0 con 0x7f70d40015e0 2023-06-28T20:07:27.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.435+0000 7fcba8cf6700 1 --2- 172.21.15.43:0/1680171515 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fcb8806adb0 0x7fcb8806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.436+0000 7fcbaaf5a700 1 --2- 172.21.15.43:0/1680171515 >> [v2:172.21.15.43:6800/2709258135,v1:172.21.15.43:6801/2709258135] conn(0x7fcb840015e0 0x7fcb84003a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.436+0000 7fcbaaf5a700 1 -- 172.21.15.43:0/1680171515 --> [v2:172.21.15.43:6800/2709258135,v1:172.21.15.43:6801/2709258135] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb84006bc0 con 0x7fcb840015e0 2023-06-28T20:07:27.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.436+0000 7fcba8cf6700 1 --2- 172.21.15.43:0/1680171515 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fcb8806adb0 0x7fcb8806d260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcba40b4be0 tx=0x7fcb98006c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.435+0000 7fdbd1845700 1 -- 172.21.15.43:0/2406547754 --> [v2:172.21.15.178:6816/3953458472,v1:172.21.15.178:6817/3953458472] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdbcc1694c0 con 0x7fdbb8073560 2023-06-28T20:07:27.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.436+0000 7fdbb3fff700 1 -- 172.21.15.43:0/2406547754 <== osd.7 v2:172.21.15.178:6816/3953458472 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdbbc00a210 con 0x7fdbb8073560 2023-06-28T20:07:27.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.437+0000 7fcba94f7700 1 --2- 172.21.15.43:0/1680171515 >> [v2:172.21.15.43:6800/2709258135,v1:172.21.15.43:6801/2709258135] conn(0x7fcb840015e0 0x7fcb84003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.437+0000 7fcba94f7700 1 --2- 172.21.15.43:0/1680171515 >> [v2:172.21.15.43:6800/2709258135,v1:172.21.15.43:6801/2709258135] conn(0x7fcb840015e0 0x7fcb84003a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.438+0000 7fdbd1845700 1 -- 172.21.15.43:0/2406547754 >> [v2:172.21.15.178:6816/3953458472,v1:172.21.15.178:6817/3953458472] conn(0x7fdbb8073560 msgr2=0x7fdbb8075970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.438+0000 7fdbd1845700 1 --2- 172.21.15.43:0/2406547754 >> [v2:172.21.15.178:6816/3953458472,v1:172.21.15.178:6817/3953458472] conn(0x7fdbb8073560 0x7fdbb8075970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.438+0000 7fcba17fa700 1 -- 172.21.15.43:0/1680171515 <== osd.0 v2:172.21.15.43:6800/2709258135 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fcb84006bc0 con 0x7fcb840015e0 2023-06-28T20:07:27.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.438+0000 7fdbd1845700 1 -- 172.21.15.43:0/2406547754 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fdbb806ad30 msgr2=0x7fdbb806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.438+0000 7fdbd1845700 1 --2- 172.21.15.43:0/2406547754 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fdbb806ad30 0x7fdbb806d1e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4000f80 tx=0x7fdbc4009000 comp rx=0 tx=0).stop 2023-06-28T20:07:27.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.438+0000 7fdbd1845700 1 -- 172.21.15.43:0/2406547754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbcc1690f0 msgr2=0x7fdbcc202730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.438+0000 7fdbd1845700 1 --2- 172.21.15.43:0/2406547754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbcc1690f0 0x7fdbcc202730 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000c370 tx=0x7fdbc000c730 comp rx=0 tx=0).stop 2023-06-28T20:07:27.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.438+0000 7fdbcb7fe700 1 -- 172.21.15.43:0/2406547754 reap_dead start 2023-06-28T20:07:27.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.439+0000 7fdbd1845700 1 -- 172.21.15.43:0/2406547754 shutdown_connections 2023-06-28T20:07:27.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.439+0000 7fdbd1845700 1 -- 172.21.15.43:0/2406547754 >> 172.21.15.43:0/2406547754 conn(0x7fdbcc069ef0 msgr2=0x7fdbcc0af9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.439+0000 7fdbd1845700 1 -- 172.21.15.43:0/2406547754 shutdown_connections 2023-06-28T20:07:27.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.439+0000 7fdbd1845700 1 -- 172.21.15.43:0/2406547754 wait complete. 2023-06-28T20:07:27.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.440+0000 7fd1626ef700 1 -- 172.21.15.43:0/3809256105 --> [v2:172.21.15.178:6820/2918661202,v1:172.21.15.178:6822/2918661202] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd13c002d00 con 0x7fd148073560 2023-06-28T20:07:27.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.441+0000 7f79f384f700 1 -- 172.21.15.43:0/440453923 --> [v2:172.21.15.178:6800/2113985646,v1:172.21.15.178:6801/2113985646] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f79ec1fa510 con 0x7f79d0073490 2023-06-28T20:07:27.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.442+0000 7f79e1ffb700 1 -- 172.21.15.43:0/440453923 <== osd.6 v2:172.21.15.178:6800/2113985646 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f79ec1fa510 con 0x7f79d0073490 2023-06-28T20:07:27.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.442+0000 7f79f384f700 1 -- 172.21.15.43:0/440453923 >> [v2:172.21.15.178:6800/2113985646,v1:172.21.15.178:6801/2113985646] conn(0x7f79d0073490 msgr2=0x7f79d00758a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.442+0000 7f79f384f700 1 --2- 172.21.15.43:0/440453923 >> [v2:172.21.15.178:6800/2113985646,v1:172.21.15.178:6801/2113985646] conn(0x7f79d0073490 0x7f79d00758a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.442+0000 7f79f384f700 1 -- 172.21.15.43:0/440453923 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f79d006ace0 msgr2=0x7f79d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.442+0000 7f79f384f700 1 --2- 172.21.15.43:0/440453923 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f79d006ace0 0x7f79d006d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f79ec0b56e0 tx=0x7f79e8009040 comp rx=0 tx=0).stop 2023-06-28T20:07:27.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.442+0000 7f79f384f700 1 -- 172.21.15.43:0/440453923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79ec0b3f70 msgr2=0x7f79ec1f5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.442+0000 7f79f384f700 1 --2- 172.21.15.43:0/440453923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f79ec0b3f70 0x7f79ec1f5ad0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f79ec06a130 tx=0x7f79dc004750 comp rx=0 tx=0).stop 2023-06-28T20:07:27.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.442+0000 7f79f1dec700 1 -- 172.21.15.43:0/440453923 reap_dead start 2023-06-28T20:07:27.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.442+0000 7f79f384f700 1 -- 172.21.15.43:0/440453923 shutdown_connections 2023-06-28T20:07:27.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.442+0000 7f79f384f700 1 -- 172.21.15.43:0/440453923 >> 172.21.15.43:0/440453923 conn(0x7f79ec09c210 msgr2=0x7f79ec0d2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.442+0000 7f79f384f700 1 -- 172.21.15.43:0/440453923 shutdown_connections 2023-06-28T20:07:27.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.442+0000 7f79f384f700 1 -- 172.21.15.43:0/440453923 wait complete. 2023-06-28T20:07:27.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.448+0000 7fd158ff9700 1 -- 172.21.15.43:0/3809256105 <== osd.4 v2:172.21.15.178:6820/2918661202 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd13c002d00 con 0x7fd148073560 2023-06-28T20:07:27.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.448+0000 7efc25f20700 1 -- 172.21.15.43:0/3495947151 --> [v2:172.21.15.43:6816/2901255345,v1:172.21.15.43:6817/2901255345] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efc200d1760 con 0x7efc0c0735e0 2023-06-28T20:07:27.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.448+0000 7f1d87a08700 1 Processor -- start 2023-06-28T20:07:27.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.449+0000 7efc07fff700 1 -- 172.21.15.43:0/3495947151 <== osd.3 v2:172.21.15.43:6816/2901255345 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efc1800f040 con 0x7efc0c0735e0 2023-06-28T20:07:27.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.448+0000 7f1d87a08700 1 -- start start 2023-06-28T20:07:27.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.450+0000 7f1d87a08700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1d8006a830 0x7f1d800c9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.450+0000 7f1d87a08700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1d8006ac00 0x7f1d8006b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.450+0000 7f1d87a08700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d800ce370 con 0x7f1d80069ef0 2023-06-28T20:07:27.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.450+0000 7f1d87a08700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d800ce610 con 0x7f1d8006a830 2023-06-28T20:07:27.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.450+0000 7f1d87a08700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d800ce8f0 con 0x7f1d8006ac00 2023-06-28T20:07:27.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.450+0000 7f1d857a4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1d8006a830 0x7f1d800c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.450+0000 7f1d857a4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1d8006a830 0x7f1d800c9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53108/0 (socket says 172.21.15.43:53108) 2023-06-28T20:07:27.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.450+0000 7f1d857a4700 1 -- 172.21.15.43:0/1610171302 learned_addr learned my addr 172.21.15.43:0/1610171302 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.450+0000 7f1d84fa3700 1 --2- 172.21.15.43:0/1610171302 >> v2:172.21.15.178:3300/0 conn(0x7f1d8006ac00 0x7f1d8006b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.451+0000 7f1d857a4700 1 -- 172.21.15.43:0/1610171302 >> v2:172.21.15.178:3300/0 conn(0x7f1d8006ac00 msgr2=0x7f1d8006b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.451+0000 7f1d857a4700 1 --2- 172.21.15.43:0/1610171302 >> v2:172.21.15.178:3300/0 conn(0x7f1d8006ac00 0x7f1d8006b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.451+0000 7f1d857a4700 1 -- 172.21.15.43:0/1610171302 >> v1:172.21.15.43:6789/0 conn(0x7f1d80069ef0 legacy=0x7f1d8006c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.451+0000 7f1d857a4700 1 -- 172.21.15.43:0/1610171302 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8006b080 con 0x7f1d8006a830 2023-06-28T20:07:27.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.451+0000 7f1d84fa3700 1 --2- 172.21.15.43:0/1610171302 >> v2:172.21.15.178:3300/0 conn(0x7f1d8006ac00 0x7f1d8006b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:27.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.451+0000 7fd1626ef700 1 -- 172.21.15.43:0/3809256105 >> [v2:172.21.15.178:6820/2918661202,v1:172.21.15.178:6822/2918661202] conn(0x7fd148073560 msgr2=0x7fd148075970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.451+0000 7fd1626ef700 1 --2- 172.21.15.43:0/3809256105 >> [v2:172.21.15.178:6820/2918661202,v1:172.21.15.178:6822/2918661202] conn(0x7fd148073560 0x7fd148075970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.451+0000 7f1d857a4700 1 --2- 172.21.15.43:0/1610171302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d8006a830 0x7f1d800c9900 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0098f0 tx=0x7f1d7c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e927ee2ad46e5914 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.451+0000 7efc25f20700 1 -- 172.21.15.43:0/3495947151 >> [v2:172.21.15.43:6816/2901255345,v1:172.21.15.43:6817/2901255345] conn(0x7efc0c0735e0 msgr2=0x7efc0c0759f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.451+0000 7efc25f20700 1 --2- 172.21.15.43:0/3495947151 >> [v2:172.21.15.43:6816/2901255345,v1:172.21.15.43:6817/2901255345] conn(0x7efc0c0735e0 0x7efc0c0759f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.451+0000 7efc25f20700 1 -- 172.21.15.43:0/3495947151 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efc0c06adb0 msgr2=0x7efc0c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.451+0000 7efc25f20700 1 --2- 172.21.15.43:0/3495947151 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efc0c06adb0 0x7efc0c06d260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efc10001eb0 tx=0x7efc1000f000 comp rx=0 tx=0).stop 2023-06-28T20:07:27.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.451+0000 7efc25f20700 1 -- 172.21.15.43:0/3495947151 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc08004850 msgr2=0x7efc200c5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.451+0000 7efc25f20700 1 --2- 172.21.15.43:0/3495947151 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efc08004850 0x7efc200c5bd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7efc1400d810 tx=0x7efc1400db20 comp rx=0 tx=0).stop 2023-06-28T20:07:27.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.451+0000 7efc1ffff700 1 -- 172.21.15.43:0/3495947151 reap_dead start 2023-06-28T20:07:27.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.451+0000 7f1d77fff700 1 -- 172.21.15.43:0/1610171302 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c00f940 con 0x7f1d8006a830 2023-06-28T20:07:27.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.451+0000 7f1d77fff700 1 -- 172.21.15.43:0/1610171302 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7c00faa0 con 0x7f1d8006a830 2023-06-28T20:07:27.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.451+0000 7f1d77fff700 1 -- 172.21.15.43:0/1610171302 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c00fe30 con 0x7f1d8006a830 2023-06-28T20:07:27.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.452+0000 7efc25f20700 1 -- 172.21.15.43:0/3495947151 shutdown_connections 2023-06-28T20:07:27.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.452+0000 7efc25f20700 1 -- 172.21.15.43:0/3495947151 >> 172.21.15.43:0/3495947151 conn(0x7efc200d05c0 msgr2=0x7efc2009ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.452+0000 7f762883d700 1 -- 172.21.15.43:0/2574685447 --> [v2:172.21.15.43:6808/894763335,v1:172.21.15.43:6809/894763335] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7608002d40 con 0x7f7604073510 2023-06-28T20:07:27.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.452+0000 7f1d87a08700 1 -- 172.21.15.43:0/1610171302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d8006a830 msgr2=0x7f1d800c9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.452+0000 7f1d87a08700 1 --2- 172.21.15.43:0/1610171302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d8006a830 0x7f1d800c9900 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0098f0 tx=0x7f1d7c009c00 comp rx=0 tx=0).stop 2023-06-28T20:07:27.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.452+0000 7efc25f20700 1 -- 172.21.15.43:0/3495947151 shutdown_connections 2023-06-28T20:07:27.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.453+0000 7efc25f20700 1 -- 172.21.15.43:0/3495947151 wait complete. 2023-06-28T20:07:27.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.453+0000 7f1d87a08700 1 -- 172.21.15.43:0/1610171302 shutdown_connections 2023-06-28T20:07:27.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.453+0000 7f1d87a08700 1 --2- 172.21.15.43:0/1610171302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d8006a830 0x7f1d800c9900 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.453+0000 7f1d87a08700 1 --2- 172.21.15.43:0/1610171302 >> v2:172.21.15.178:3300/0 conn(0x7f1d8006ac00 0x7f1d8006b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.453+0000 7f1d87a08700 1 -- 172.21.15.43:0/1610171302 >> 172.21.15.43:0/1610171302 conn(0x7f1d800b1a50 msgr2=0x7f1d800cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.453+0000 7f1d87a08700 1 -- 172.21.15.43:0/1610171302 shutdown_connections 2023-06-28T20:07:27.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.453+0000 7f1d87a08700 1 -- 172.21.15.43:0/1610171302 wait complete. 2023-06-28T20:07:27.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.455+0000 7fd1626ef700 1 -- 172.21.15.43:0/3809256105 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd14806ad30 msgr2=0x7fd14806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.455+0000 7fd1626ef700 1 --2- 172.21.15.43:0/3809256105 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd14806ad30 0x7fd14806d1e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd14c00d370 tx=0x7fd14c014000 comp rx=0 tx=0).stop 2023-06-28T20:07:27.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.455+0000 7fd1626ef700 1 -- 172.21.15.43:0/3809256105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd15c1592c0 msgr2=0x7fd15c153cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.455+0000 7fd1626ef700 1 --2- 172.21.15.43:0/3809256105 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd15c1592c0 0x7fd15c153cf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd15000c4e0 tx=0x7fd15000c7f0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.455+0000 7f70f9108700 1 -- 172.21.15.43:0/4196329587 --> [v2:172.21.15.43:6824/1523179953,v1:172.21.15.43:6825/1523179953] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f70d4005ca0 con 0x7f70d40015e0 2023-06-28T20:07:27.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.455+0000 7f1d87a08700 1 Processor -- start 2023-06-28T20:07:27.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.455+0000 7fd160c8c700 1 -- 172.21.15.43:0/3809256105 reap_dead start 2023-06-28T20:07:27.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.455+0000 7f7612ffd700 1 -- 172.21.15.43:0/2574685447 <== osd.1 v2:172.21.15.43:6808/894763335 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7608002d40 con 0x7f7604073510 2023-06-28T20:07:27.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.455+0000 7fd1626ef700 1 -- 172.21.15.43:0/3809256105 shutdown_connections 2023-06-28T20:07:27.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.455+0000 7fd1626ef700 1 -- 172.21.15.43:0/3809256105 >> 172.21.15.43:0/3809256105 conn(0x7fd15c069ef0 msgr2=0x7fd15c0cb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.455+0000 7fd1626ef700 1 -- 172.21.15.43:0/3809256105 shutdown_connections 2023-06-28T20:07:27.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.455+0000 7fd1626ef700 1 -- 172.21.15.43:0/3809256105 wait complete. 2023-06-28T20:07:27.519 INFO:teuthology.orchestra.run.smithi043.stdout:55834574855 2023-06-28T20:07:27.519 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-06-28T20:07:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.455+0000 7f1d87a08700 1 -- start start 2023-06-28T20:07:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.456+0000 7f1d87a08700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d80069ef0 0x7f1d8006c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.456+0000 7f1d87a08700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d8006a830 0x7f1d800b4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.456+0000 7f1d87a08700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d8006ac00 0x7f1d8015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.456+0000 7f1d87a08700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d80153f80 con 0x7f1d8006ac00 2023-06-28T20:07:27.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.456+0000 7f1d87a08700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d800c5080 con 0x7f1d80069ef0 2023-06-28T20:07:27.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.456+0000 7f1d87a08700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d800c5360 con 0x7f1d8006a830 2023-06-28T20:07:27.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.456+0000 7f1d85fa5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d8006ac00 0x7f1d8015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.456+0000 7f1d85fa5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d8006ac00 0x7f1d8015a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53118/0 (socket says 172.21.15.43:53118) 2023-06-28T20:07:27.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.456+0000 7f1d84fa3700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d8006a830 0x7f1d800b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.456+0000 7f1d85fa5700 1 -- 172.21.15.43:0/1318634343 learned_addr learned my addr 172.21.15.43:0/1318634343 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.457+0000 7f1d85fa5700 1 -- 172.21.15.43:0/1318634343 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d8006a830 msgr2=0x7f1d800b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.457+0000 7f1d85fa5700 1 --2- 172.21.15.43:0/1318634343 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1d8006a830 0x7f1d800b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.457+0000 7f1d85fa5700 1 -- 172.21.15.43:0/1318634343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d80069ef0 msgr2=0x7f1d8006c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.457+0000 7f1d85fa5700 1 --2- 172.21.15.43:0/1318634343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d80069ef0 0x7f1d8006c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.457+0000 7f1d85fa5700 1 -- 172.21.15.43:0/1318634343 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c009550 con 0x7f1d8006ac00 2023-06-28T20:07:27.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.457+0000 7f762883d700 1 -- 172.21.15.43:0/2574685447 >> [v2:172.21.15.43:6808/894763335,v1:172.21.15.43:6809/894763335] conn(0x7f7604073510 msgr2=0x7f7604075920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.457+0000 7f762883d700 1 --2- 172.21.15.43:0/2574685447 >> [v2:172.21.15.43:6808/894763335,v1:172.21.15.43:6809/894763335] conn(0x7f7604073510 0x7f7604075920 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.457+0000 7f762883d700 1 -- 172.21.15.43:0/2574685447 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f760406ace0 msgr2=0x7f760406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.457+0000 7f762883d700 1 --2- 172.21.15.43:0/2574685447 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f760406ace0 0x7f760406d190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f76240b2f70 tx=0x7f761c00af90 comp rx=0 tx=0).stop 2023-06-28T20:07:27.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.457+0000 7f762883d700 1 -- 172.21.15.43:0/2574685447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76241ed690 msgr2=0x7f76240b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.457+0000 7f762883d700 1 --2- 172.21.15.43:0/2574685447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76241ed690 0x7f76240b41c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7618011630 tx=0x7f76180119f0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.457+0000 7f7622d9d700 1 -- 172.21.15.43:0/2574685447 reap_dead start 2023-06-28T20:07:27.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.457+0000 7f1d85fa5700 1 --2- 172.21.15.43:0/1318634343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d8006ac00 0x7f1d8015a1a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800cbc0 tx=0x7f1d7800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.457+0000 7f762883d700 1 -- 172.21.15.43:0/2574685447 shutdown_connections 2023-06-28T20:07:27.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.457+0000 7f762883d700 1 -- 172.21.15.43:0/2574685447 >> 172.21.15.43:0/2574685447 conn(0x7f7624069ef0 msgr2=0x7f7624151410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.458+0000 7f1d75ffb700 1 -- 172.21.15.43:0/1318634343 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78015dd0 con 0x7f1d8006ac00 2023-06-28T20:07:27.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.458+0000 7f1d75ffb700 1 -- 172.21.15.43:0/1318634343 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d78003a00 con 0x7f1d8006ac00 2023-06-28T20:07:27.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.458+0000 7f1d87a08700 1 -- 172.21.15.43:0/1318634343 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d800c5640 con 0x7f1d8006ac00 2023-06-28T20:07:27.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.458+0000 7f1d75ffb700 1 -- 172.21.15.43:0/1318634343 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d78003db0 con 0x7f1d8006ac00 2023-06-28T20:07:27.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.458+0000 7f1d87a08700 1 -- 172.21.15.43:0/1318634343 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d800c5c00 con 0x7f1d8006ac00 2023-06-28T20:07:27.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.459+0000 7f70db7fe700 1 -- 172.21.15.43:0/4196329587 <== osd.2 v2:172.21.15.43:6824/1523179953 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f70d4005ca0 con 0x7f70d40015e0 2023-06-28T20:07:27.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.459+0000 7f1d87a08700 1 -- 172.21.15.43:0/1318634343 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1d800ceeb0 con 0x7f1d8006ac00 2023-06-28T20:07:27.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.460+0000 7f1d75ffb700 1 -- 172.21.15.43:0/1318634343 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1d7800a680 con 0x7f1d8006ac00 2023-06-28T20:07:27.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.461+0000 7f762883d700 1 -- 172.21.15.43:0/2574685447 shutdown_connections 2023-06-28T20:07:27.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.461+0000 7f1d75ffb700 1 --2- 172.21.15.43:0/1318634343 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1d6406ad30 0x7f1d6406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.529 INFO:teuthology.orchestra.run.smithi043.stdout:55834574853 2023-06-28T20:07:27.529 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-06-28T20:07:27.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.461+0000 7f1d75ffb700 1 -- 172.21.15.43:0/1318634343 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1d780916d0 con 0x7f1d8006ac00 2023-06-28T20:07:27.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.461+0000 7f1d857a4700 1 --2- 172.21.15.43:0/1318634343 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1d6406ad30 0x7f1d6406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.461+0000 7f762883d700 1 -- 172.21.15.43:0/2574685447 wait complete. 2023-06-28T20:07:27.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.461+0000 7f1d75ffb700 1 --2- 172.21.15.43:0/1318634343 >> [v2:172.21.15.178:6808/3916491871,v1:172.21.15.178:6809/3916491871] conn(0x7f1d640735e0 0x7f1d640759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.461+0000 7f1d75ffb700 1 -- 172.21.15.43:0/1318634343 --> [v2:172.21.15.178:6808/3916491871,v1:172.21.15.178:6809/3916491871] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d640760a0 con 0x7f1d640735e0 2023-06-28T20:07:27.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.461+0000 7f1d75ffb700 1 -- 172.21.15.43:0/1318634343 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d78093de0 con 0x7f1d8006ac00 2023-06-28T20:07:27.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.461+0000 7f1d857a4700 1 --2- 172.21.15.43:0/1318634343 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1d6406ad30 0x7f1d6406d1e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c000c00 tx=0x7f1d7c012000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.461+0000 7f1d84fa3700 1 --2- 172.21.15.43:0/1318634343 >> [v2:172.21.15.178:6808/3916491871,v1:172.21.15.178:6809/3916491871] conn(0x7f1d640735e0 0x7f1d640759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.462+0000 7f1d84fa3700 1 --2- 172.21.15.43:0/1318634343 >> [v2:172.21.15.178:6808/3916491871,v1:172.21.15.178:6809/3916491871] conn(0x7f1d640735e0 0x7f1d640759f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.462+0000 7f70f9108700 1 -- 172.21.15.43:0/4196329587 >> [v2:172.21.15.43:6824/1523179953,v1:172.21.15.43:6825/1523179953] conn(0x7f70d40015e0 msgr2=0x7f70d4003a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.462+0000 7f70f9108700 1 --2- 172.21.15.43:0/4196329587 >> [v2:172.21.15.43:6824/1523179953,v1:172.21.15.43:6825/1523179953] conn(0x7f70d40015e0 0x7f70d4003a90 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.462+0000 7f70f9108700 1 -- 172.21.15.43:0/4196329587 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f70e006ace0 msgr2=0x7f70e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.462+0000 7f70f9108700 1 --2- 172.21.15.43:0/4196329587 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f70e006ace0 0x7f70e006d190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f70e4001580 tx=0x7f70e4007a20 comp rx=0 tx=0).stop 2023-06-28T20:07:27.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.462+0000 7f70f9108700 1 -- 172.21.15.43:0/4196329587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f40cfe00 msgr2=0x7f70f40c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.462+0000 7f70f9108700 1 --2- 172.21.15.43:0/4196329587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70f40cfe00 0x7f70f40c8f50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f70f406bf40 tx=0x7f70ec004ad0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.462+0000 7f70f359e700 1 -- 172.21.15.43:0/4196329587 reap_dead start 2023-06-28T20:07:27.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.462+0000 7f70f9108700 1 -- 172.21.15.43:0/4196329587 shutdown_connections 2023-06-28T20:07:27.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.462+0000 7f70f9108700 1 -- 172.21.15.43:0/4196329587 >> 172.21.15.43:0/4196329587 conn(0x7f70f40664b0 msgr2=0x7f70f406d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.463+0000 7f1d75ffb700 1 -- 172.21.15.43:0/1318634343 <== osd.5 v2:172.21.15.178:6808/3916491871 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1d640760a0 con 0x7f1d640735e0 2023-06-28T20:07:27.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.466+0000 7f70f9108700 1 -- 172.21.15.43:0/4196329587 shutdown_connections 2023-06-28T20:07:27.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.466+0000 7f70f9108700 1 -- 172.21.15.43:0/4196329587 wait complete. 2023-06-28T20:07:27.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.468+0000 7fcbaaf5a700 1 -- 172.21.15.43:0/1680171515 --> [v2:172.21.15.43:6800/2709258135,v1:172.21.15.43:6801/2709258135] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcb84005c40 con 0x7fcb840015e0 2023-06-28T20:07:27.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.468+0000 7fcba17fa700 1 -- 172.21.15.43:0/1680171515 <== osd.0 v2:172.21.15.43:6800/2709258135 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcb84005c40 con 0x7fcb840015e0 2023-06-28T20:07:27.536 INFO:teuthology.orchestra.run.smithi043.stdout:55834574853 2023-06-28T20:07:27.536 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-06-28T20:07:27.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.476+0000 7fcbaaf5a700 1 -- 172.21.15.43:0/1680171515 >> [v2:172.21.15.43:6800/2709258135,v1:172.21.15.43:6801/2709258135] conn(0x7fcb840015e0 msgr2=0x7fcb84003a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.476+0000 7fcbaaf5a700 1 --2- 172.21.15.43:0/1680171515 >> [v2:172.21.15.43:6800/2709258135,v1:172.21.15.43:6801/2709258135] conn(0x7fcb840015e0 0x7fcb84003a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.476+0000 7fcbaaf5a700 1 -- 172.21.15.43:0/1680171515 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fcb8806adb0 msgr2=0x7fcb8806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.476+0000 7fcbaaf5a700 1 --2- 172.21.15.43:0/1680171515 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fcb8806adb0 0x7fcb8806d260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcba40b4be0 tx=0x7fcb98006c00 comp rx=0 tx=0).stop 2023-06-28T20:07:27.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.476+0000 7fcbaaf5a700 1 -- 172.21.15.43:0/1680171515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba40ce290 msgr2=0x7fcba41ff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.476+0000 7fcbaaf5a700 1 --2- 172.21.15.43:0/1680171515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcba40ce290 0x7fcba41ff090 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000c4d0 tx=0x7fcb9000c890 comp rx=0 tx=0).stop 2023-06-28T20:07:27.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.477+0000 7fcba94f7700 1 -- 172.21.15.43:0/1680171515 reap_dead start 2023-06-28T20:07:27.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.477+0000 7fcbaaf5a700 1 -- 172.21.15.43:0/1680171515 shutdown_connections 2023-06-28T20:07:27.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.477+0000 7fcbaaf5a700 1 -- 172.21.15.43:0/1680171515 >> 172.21.15.43:0/1680171515 conn(0x7fcba4069ef0 msgr2=0x7fcba40cab10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.539 INFO:teuthology.orchestra.run.smithi043.stdout:55834574853 2023-06-28T20:07:27.539 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-06-28T20:07:27.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.478+0000 7fcbaaf5a700 1 -- 172.21.15.43:0/1680171515 shutdown_connections 2023-06-28T20:07:27.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.478+0000 7fcbaaf5a700 1 -- 172.21.15.43:0/1680171515 wait complete. 2023-06-28T20:07:27.541 INFO:teuthology.orchestra.run.smithi043.stdout:55834574852 2023-06-28T20:07:27.541 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-06-28T20:07:27.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.481+0000 7f1d87a08700 1 -- 172.21.15.43:0/1318634343 --> [v2:172.21.15.178:6808/3916491871,v1:172.21.15.178:6809/3916491871] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1d800c4e70 con 0x7f1d640735e0 2023-06-28T20:07:27.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.482+0000 7f1d75ffb700 1 -- 172.21.15.43:0/1318634343 <== osd.5 v2:172.21.15.178:6808/3916491871 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1d800c4e70 con 0x7f1d640735e0 2023-06-28T20:07:27.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.482+0000 7f1d87a08700 1 -- 172.21.15.43:0/1318634343 >> [v2:172.21.15.178:6808/3916491871,v1:172.21.15.178:6809/3916491871] conn(0x7f1d640735e0 msgr2=0x7f1d640759f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.482+0000 7f1d87a08700 1 --2- 172.21.15.43:0/1318634343 >> [v2:172.21.15.178:6808/3916491871,v1:172.21.15.178:6809/3916491871] conn(0x7f1d640735e0 0x7f1d640759f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.482+0000 7f1d87a08700 1 -- 172.21.15.43:0/1318634343 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1d6406ad30 msgr2=0x7f1d6406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.482+0000 7f1d87a08700 1 --2- 172.21.15.43:0/1318634343 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1d6406ad30 0x7f1d6406d1e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c000c00 tx=0x7f1d7c012000 comp rx=0 tx=0).stop 2023-06-28T20:07:27.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.482+0000 7f1d87a08700 1 -- 172.21.15.43:0/1318634343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d8006ac00 msgr2=0x7f1d8015a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.482+0000 7f1d87a08700 1 --2- 172.21.15.43:0/1318634343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d8006ac00 0x7f1d8015a1a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1d7800cbc0 tx=0x7f1d7800cf80 comp rx=0 tx=0).stop 2023-06-28T20:07:27.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.482+0000 7f1d85fa5700 1 -- 172.21.15.43:0/1318634343 reap_dead start 2023-06-28T20:07:27.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.482+0000 7f1d87a08700 1 -- 172.21.15.43:0/1318634343 shutdown_connections 2023-06-28T20:07:27.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.482+0000 7f1d87a08700 1 -- 172.21.15.43:0/1318634343 >> 172.21.15.43:0/1318634343 conn(0x7f1d800b1a50 msgr2=0x7f1d80154b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.482+0000 7f1d87a08700 1 -- 172.21.15.43:0/1318634343 shutdown_connections 2023-06-28T20:07:27.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.482+0000 7f1d87a08700 1 -- 172.21.15.43:0/1318634343 wait complete. 2023-06-28T20:07:27.545 INFO:teuthology.orchestra.run.smithi043.stdout:55834574853 2023-06-28T20:07:27.545 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-06-28T20:07:27.546 INFO:teuthology.orchestra.run.smithi043.stdout:55834574853 2023-06-28T20:07:27.546 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-06-28T20:07:27.547 INFO:teuthology.orchestra.run.smithi043.stdout:55834574854 2023-06-28T20:07:27.548 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-06-28T20:07:27.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.781+0000 7f5455533700 1 Processor -- start 2023-06-28T20:07:27.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.782+0000 7f5455533700 1 -- start start 2023-06-28T20:07:27.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.783+0000 7f5455533700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5450158ef0 0x7f545006cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.783+0000 7f5455533700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54501592c0 0x7f54500ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.783+0000 7f5455533700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5450159710 0x7f54500ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.783+0000 7f5455533700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54500caf00 con 0x7f5450159710 2023-06-28T20:07:27.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.783+0000 7f5455533700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f545006d610 con 0x7f5450158ef0 2023-06-28T20:07:27.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.783+0000 7f5455533700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545015c110 con 0x7f54501592c0 2023-06-28T20:07:27.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.783+0000 7f544f7fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5450159710 0x7f54500ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.783+0000 7f544f7fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5450159710 0x7f54500ca9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53128/0 (socket says 172.21.15.43:53128) 2023-06-28T20:07:27.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.783+0000 7f544f7fe700 1 -- 172.21.15.43:0/3546358058 learned_addr learned my addr 172.21.15.43:0/3546358058 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.784+0000 7f544f7fe700 1 -- 172.21.15.43:0/3546358058 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54501592c0 msgr2=0x7f54500ca480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-28T20:07:27.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.784+0000 7f544f7fe700 1 --2- 172.21.15.43:0/3546358058 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54501592c0 0x7f54500ca480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.784+0000 7f544f7fe700 1 -- 172.21.15.43:0/3546358058 >> v2:172.21.15.178:3300/0 conn(0x7f5450158ef0 msgr2=0x7f545006cf10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-28T20:07:27.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.784+0000 7f544f7fe700 1 --2- 172.21.15.43:0/3546358058 >> v2:172.21.15.178:3300/0 conn(0x7f5450158ef0 0x7f545006cf10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.784+0000 7f544f7fe700 1 -- 172.21.15.43:0/3546358058 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545015c3f0 con 0x7f5450159710 2023-06-28T20:07:27.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.784+0000 7f544f7fe700 1 --2- 172.21.15.43:0/3546358058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5450159710 0x7f54500ca9c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f54500cb3f0 tx=0x7f5440008bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c79a8defccc2d2 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.787+0000 7f544dffb700 1 -- 172.21.15.43:0/3546358058 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544000b820 con 0x7f5450159710 2023-06-28T20:07:27.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.787+0000 7f544dffb700 1 -- 172.21.15.43:0/3546358058 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544000b980 con 0x7f5450159710 2023-06-28T20:07:27.791 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.787+0000 7f544dffb700 1 -- 172.21.15.43:0/3546358058 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54400126f0 con 0x7f5450159710 2023-06-28T20:07:27.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.788+0000 7f5455533700 1 -- 172.21.15.43:0/3546358058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5450159710 msgr2=0x7f54500ca9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.788+0000 7f5455533700 1 --2- 172.21.15.43:0/3546358058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5450159710 0x7f54500ca9c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f54500cb3f0 tx=0x7f5440008bd0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.788+0000 7fb9d2004700 1 Processor -- start 2023-06-28T20:07:27.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.788+0000 7f17c078a700 1 Processor -- start 2023-06-28T20:07:27.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.788+0000 7f17c078a700 1 -- start start 2023-06-28T20:07:27.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.788+0000 7f5455533700 1 -- 172.21.15.43:0/3546358058 shutdown_connections 2023-06-28T20:07:27.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.788+0000 7f5455533700 1 --2- 172.21.15.43:0/3546358058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5450159710 0x7f54500ca9c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.788+0000 7f5455533700 1 --2- 172.21.15.43:0/3546358058 >> v2:172.21.15.178:3300/0 conn(0x7f5450158ef0 0x7f545006cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.788+0000 7f5455533700 1 --2- 172.21.15.43:0/3546358058 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54501592c0 0x7f54500ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.788+0000 7f5455533700 1 -- 172.21.15.43:0/3546358058 >> 172.21.15.43:0/3546358058 conn(0x7f5450069ef0 msgr2=0x7f545006a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.788+0000 7fb9d2004700 1 -- start start 2023-06-28T20:07:27.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.789+0000 7ff57a0ec700 1 Processor -- start 2023-06-28T20:07:27.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.789+0000 7f6746a68700 1 Processor -- start 2023-06-28T20:07:27.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.789+0000 7f6746a68700 1 -- start start 2023-06-28T20:07:27.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7fb9d2004700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb9cc06a9d0 0x7fb9cc06cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7fb9d2004700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9cc0ca480 0x7fb9cc0ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7fb9d2004700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb9cc158ef0 0x7fb9cc159370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7fb9d2004700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9cc06d610 con 0x7fb9cc06a9d0 2023-06-28T20:07:27.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7fb9d2004700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9cc15c110 con 0x7fb9cc158ef0 2023-06-28T20:07:27.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7fb9d2004700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc15c3f0 con 0x7fb9cc0ca480 2023-06-28T20:07:27.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7fb9cb7fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb9cc06a9d0 0x7fb9cc06cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7fb9cbfff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb9cc158ef0 0x7fb9cc159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7fb9cb7fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb9cc06a9d0 0x7fb9cc06cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53136/0 (socket says 172.21.15.43:53136) 2023-06-28T20:07:27.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7fb9cb7fe700 1 -- 172.21.15.43:0/604936165 learned_addr learned my addr 172.21.15.43:0/604936165 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.789+0000 7f17c078a700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f17b808a660 0x7f17b80cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.789+0000 7f17c078a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b80cf0b0 0x7f17b80ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.789+0000 7f17c078a700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b816cfb0 con 0x7f17b80cde80 2023-06-28T20:07:27.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.789+0000 7f17c078a700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17b816dc30 con 0x7f17b808a660 2023-06-28T20:07:27.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.789+0000 7f17c078a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b816d290 con 0x7f17b80cf0b0 2023-06-28T20:07:27.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7f17bed27700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b80cf0b0 0x7f17b80ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7f17bed27700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b80cf0b0 0x7f17b80ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.43:37102/0 (socket says 172.21.15.43:37102) 2023-06-28T20:07:27.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7f17bed27700 1 -- 172.21.15.43:0/1118029552 learned_addr learned my addr 172.21.15.43:0/1118029552 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7f17be526700 1 --2- 172.21.15.43:0/1118029552 >> v2:172.21.15.178:3300/0 conn(0x7f17b808a660 0x7f17b80cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.789+0000 7ff57a0ec700 1 -- start start 2023-06-28T20:07:27.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7f17bed27700 1 -- 172.21.15.43:0/1118029552 >> v2:172.21.15.178:3300/0 conn(0x7f17b808a660 msgr2=0x7f17b80cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7f17bed27700 1 --2- 172.21.15.43:0/1118029552 >> v2:172.21.15.178:3300/0 conn(0x7f17b808a660 0x7f17b80cd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7f17bed27700 1 -- 172.21.15.43:0/1118029552 >> v1:172.21.15.178:6789/0 conn(0x7f17b80cde80 legacy=0x7f17b80ce9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7f17bed27700 1 -- 172.21.15.43:0/1118029552 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b816d570 con 0x7f17b80cf0b0 2023-06-28T20:07:27.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7fb9cbfff700 1 -- 172.21.15.43:0/604936165 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9cc0ca480 msgr2=0x7fb9cc0ca8d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-28T20:07:27.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7fb9cbfff700 1 --2- 172.21.15.43:0/604936165 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9cc0ca480 0x7fb9cc0ca8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7fb9cbfff700 1 -- 172.21.15.43:0/604936165 >> v2:172.21.15.43:3300/0 conn(0x7fb9cc06a9d0 msgr2=0x7fb9cc06cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7fb9cbfff700 1 --2- 172.21.15.43:0/604936165 >> v2:172.21.15.43:3300/0 conn(0x7fb9cc06a9d0 0x7fb9cc06cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7fb9cbfff700 1 -- 172.21.15.43:0/604936165 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9cc15c6d0 con 0x7fb9cc158ef0 2023-06-28T20:07:27.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7f6746a68700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f674006d8b0 0x7f674006cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7fb9cb7fe700 1 --2- 172.21.15.43:0/604936165 >> v2:172.21.15.43:3300/0 conn(0x7fb9cc06a9d0 0x7fb9cc06cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:27.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.790+0000 7f17bed27700 1 --2- 172.21.15.43:0/1118029552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b80cf0b0 0x7f17b80ca480 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f17b0014a00 tx=0x7f17b0014d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6e24087a46ec673 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7f6746a68700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67401594c0 con 0x7f67400ca480 2023-06-28T20:07:27.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7f6746a68700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f674006d610 con 0x7f674015c470 2023-06-28T20:07:27.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7f6746a68700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6740158b90 con 0x7f674006d8b0 2023-06-28T20:07:27.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7fb9cbfff700 1 --2- 172.21.15.43:0/604936165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9cc158ef0 0x7fb9cc159370 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc0cf590 tx=0x7fb9bc009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb41e7aada9d4c4b server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7ff57a0ec700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff5740b0130 0x7ff574158b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7ff57a0ec700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5740cccb0 con 0x7ff5740c9400 2023-06-28T20:07:27.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7ff57a0ec700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5740ae6c0 con 0x7ff574066e90 2023-06-28T20:07:27.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7ff57a0ec700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5740aea60 con 0x7ff5740b0130 2023-06-28T20:07:27.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7f17bd524700 1 -- 172.21.15.43:0/1118029552 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b0014ea0 con 0x7f17b80cf0b0 2023-06-28T20:07:27.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7f17bd524700 1 -- 172.21.15.43:0/1118029552 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b0003c20 con 0x7f17b80cf0b0 2023-06-28T20:07:27.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7f17bd524700 1 -- 172.21.15.43:0/1118029552 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b000f1c0 con 0x7f17b80cf0b0 2023-06-28T20:07:27.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7ff5737fe700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7ff574066e90 0x7ff574152a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:41454/0 (socket says 172.21.15.43:41454) 2023-06-28T20:07:27.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7ff5737fe700 1 -- 172.21.15.43:0/2270509348 learned_addr learned my addr 172.21.15.43:0/2270509348 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:27.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7f673ffff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f67400ca480 0x7f67400ca8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42328/0 (socket says 172.21.15.43:42328) 2023-06-28T20:07:27.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7f673ffff700 1 -- 172.21.15.43:0/1912407117 learned_addr learned my addr 172.21.15.43:0/1912407117 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:27.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7f5455533700 1 -- 172.21.15.43:0/3546358058 shutdown_connections 2023-06-28T20:07:27.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7fb9ca7fc700 1 -- 172.21.15.43:0/604936165 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc011830 con 0x7fb9cc158ef0 2023-06-28T20:07:27.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7fb9ca7fc700 1 -- 172.21.15.43:0/604936165 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9bc011990 con 0x7fb9cc158ef0 2023-06-28T20:07:27.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7fb9ca7fc700 1 -- 172.21.15.43:0/604936165 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc011d20 con 0x7fb9cc158ef0 2023-06-28T20:07:27.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7f66c9145700 1 Processor -- start 2023-06-28T20:07:27.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7f66c9145700 1 -- start start 2023-06-28T20:07:27.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7ff572ffd700 1 --2- 172.21.15.43:0/2270509348 >> v2:172.21.15.43:3300/0 conn(0x7ff5740b0130 0x7ff574158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7f673f7fe700 1 -- 172.21.15.43:0/1912407117 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1083977063 0 0) 0x7f67401594c0 con 0x7f67400ca480 2023-06-28T20:07:27.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7f673f7fe700 1 -- 172.21.15.43:0/1912407117 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67280035b0 con 0x7f67400ca480 2023-06-28T20:07:27.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.791+0000 7f5455533700 1 -- 172.21.15.43:0/3546358058 wait complete. 2023-06-28T20:07:27.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.792+0000 7f17c078a700 1 -- 172.21.15.43:0/1118029552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b80cf0b0 msgr2=0x7f17b80ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.792+0000 7f17c078a700 1 --2- 172.21.15.43:0/1118029552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b80cf0b0 0x7f17b80ca480 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f17b0014a00 tx=0x7f17b0014d10 comp rx=0 tx=0).stop 2023-06-28T20:07:27.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.792+0000 7fb9d2004700 1 -- 172.21.15.43:0/604936165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9cc158ef0 msgr2=0x7fb9cc159370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.792+0000 7fb9d2004700 1 --2- 172.21.15.43:0/604936165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9cc158ef0 0x7fb9cc159370 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc0cf590 tx=0x7fb9bc009b20 comp rx=0 tx=0).stop 2023-06-28T20:07:27.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.792+0000 7f17c078a700 1 -- 172.21.15.43:0/1118029552 shutdown_connections 2023-06-28T20:07:27.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.792+0000 7f17c078a700 1 --2- 172.21.15.43:0/1118029552 >> v2:172.21.15.178:3300/0 conn(0x7f17b808a660 0x7f17b80cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.792+0000 7f17c078a700 1 --2- 172.21.15.43:0/1118029552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b80cf0b0 0x7f17b80ca480 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.792+0000 7f17c078a700 1 -- 172.21.15.43:0/1118029552 >> 172.21.15.43:0/1118029552 conn(0x7f17b8069ef0 msgr2=0x7f17b806a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.792+0000 7f6744804700 1 --2- 172.21.15.43:0/1912407117 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f674006d8b0 0x7f674006cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.792+0000 7f673f7fe700 1 -- 172.21.15.43:0/1912407117 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2662960684 0 0) 0x7f67280035b0 con 0x7f67400ca480 2023-06-28T20:07:27.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.792+0000 7f673f7fe700 1 -- 172.21.15.43:0/1912407117 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f674006d8b0 msgr2=0x7f674006cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.792+0000 7f673f7fe700 1 --2- 172.21.15.43:0/1912407117 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f674006d8b0 0x7f674006cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.792+0000 7f673f7fe700 1 -- 172.21.15.43:0/1912407117 >> v1:172.21.15.178:6789/0 conn(0x7f674015c470 legacy=0x7f674015c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.792+0000 7f673f7fe700 1 -- 172.21.15.43:0/1912407117 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6740158e70 con 0x7f67400ca480 2023-06-28T20:07:27.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.792+0000 7f673f7fe700 1 -- 172.21.15.43:0/1912407117 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f67380019f0 con 0x7f67400ca480 2023-06-28T20:07:27.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.792+0000 7f673f7fe700 1 -- 172.21.15.43:0/1912407117 >> v1:172.21.15.43:6789/0 conn(0x7f67400ca480 legacy=0x7f67400ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.792+0000 7f673f7fe700 1 --2- 172.21.15.43:0/1912407117 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f674006d8b0 0x7f6728003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.792+0000 7f673f7fe700 1 --2- 172.21.15.43:0/1912407117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6728004040 0x7f6728004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.792+0000 7f07e57f6700 1 Processor -- start 2023-06-28T20:07:27.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.792+0000 7f673f7fe700 1 --2- 172.21.15.43:0/1912407117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6728004d90 0x7f6728005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f673f7fe700 1 -- 172.21.15.43:0/1912407117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6740158b90 con 0x7f6728004d90 2023-06-28T20:07:27.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f673f7fe700 1 -- 172.21.15.43:0/1912407117 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672800bba0 con 0x7f6728004040 2023-06-28T20:07:27.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f673f7fe700 1 -- 172.21.15.43:0/1912407117 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672800bd60 con 0x7f674006d8b0 2023-06-28T20:07:27.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f6744804700 1 --2- 172.21.15.43:0/1912407117 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f674006d8b0 0x7f6728003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f6745005700 1 --2- 172.21.15.43:0/1912407117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6728004d90 0x7f6728005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f673ffff700 1 --2- 172.21.15.43:0/1912407117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6728004040 0x7f6728004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f66c9145700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66c40cccf0 0x7f66c4167490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f66c9145700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c40cf2c0 con 0x7f66c4142930 2023-06-28T20:07:27.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f66c9145700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c40ce6f0 con 0x7f66c40cbb10 2023-06-28T20:07:27.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f66c9145700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c4142d00 con 0x7f66c40cccf0 2023-06-28T20:07:27.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f07e57f6700 1 -- start start 2023-06-28T20:07:27.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f6744804700 1 -- 172.21.15.43:0/1912407117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6728004040 msgr2=0x7f6728004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f6744804700 1 --2- 172.21.15.43:0/1912407117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6728004040 0x7f6728004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f6744804700 1 -- 172.21.15.43:0/1912407117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6728004d90 msgr2=0x7f6728005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f6744804700 1 --2- 172.21.15.43:0/1912407117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6728004d90 0x7f6728005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f6744804700 1 -- 172.21.15.43:0/1912407117 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6740158e70 con 0x7f674006d8b0 2023-06-28T20:07:27.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f66c259c700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f66c40cbb10 0x7f66c40cc5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:41470/0 (socket says 172.21.15.43:41470) 2023-06-28T20:07:27.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f66c259c700 1 -- 172.21.15.43:0/718200618 learned_addr learned my addr 172.21.15.43:0/718200618 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f6744804700 1 --2- 172.21.15.43:0/1912407117 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f674006d8b0 0x7f6728003a30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f673000ec20 tx=0x7f6728005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c1108e9747dc717 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f673f7fe700 1 -- 172.21.15.43:0/1912407117 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6730005c20 con 0x7f674006d8b0 2023-06-28T20:07:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f673f7fe700 1 -- 172.21.15.43:0/1912407117 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6730005d80 con 0x7f674006d8b0 2023-06-28T20:07:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f673f7fe700 1 -- 172.21.15.43:0/1912407117 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6730010b50 con 0x7f674006d8b0 2023-06-28T20:07:27.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f5455533700 1 Processor -- start 2023-06-28T20:07:27.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f66c1d9b700 1 -- 172.21.15.43:0/718200618 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2725631184 0 0) 0x7f66c40cf2c0 con 0x7f66c4142930 2023-06-28T20:07:27.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f66c1d9b700 1 -- 172.21.15.43:0/718200618 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66ac0035b0 con 0x7f66c4142930 2023-06-28T20:07:27.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f66c1d9b700 1 -- 172.21.15.43:0/718200618 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2653536989 0 0) 0x7f66c40ce6f0 con 0x7f66c40cbb10 2023-06-28T20:07:27.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f66c1d9b700 1 -- 172.21.15.43:0/718200618 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66c40cf2c0 con 0x7f66c40cbb10 2023-06-28T20:07:27.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f66c359e700 1 --2- 172.21.15.43:0/718200618 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66c40cccf0 0x7f66c4167490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7ff572ffd700 1 -- 172.21.15.43:0/2270509348 >> v1:172.21.15.178:6789/0 conn(0x7ff574066e90 legacy=0x7ff574152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7ff572ffd700 1 -- 172.21.15.43:0/2270509348 >> v1:172.21.15.43:6789/0 conn(0x7ff5740c9400 legacy=0x7ff574069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7ff572ffd700 1 -- 172.21.15.43:0/2270509348 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff574159420 con 0x7ff5740b0130 2023-06-28T20:07:27.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f17c078a700 1 -- 172.21.15.43:0/1118029552 shutdown_connections 2023-06-28T20:07:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f66c1d9b700 1 -- 172.21.15.43:0/718200618 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2631247909 0 0) 0x7f66ac0035b0 con 0x7f66c4142930 2023-06-28T20:07:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f66c1d9b700 1 -- 172.21.15.43:0/718200618 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66c40cccf0 msgr2=0x7f66c4167490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f66c1d9b700 1 --2- 172.21.15.43:0/718200618 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66c40cccf0 0x7f66c4167490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f66c1d9b700 1 -- 172.21.15.43:0/718200618 >> v1:172.21.15.178:6789/0 conn(0x7f66c40cbb10 legacy=0x7f66c40cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f66c1d9b700 1 -- 172.21.15.43:0/718200618 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c406cdb0 con 0x7f66c4142930 2023-06-28T20:07:27.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f66c1d9b700 1 -- 172.21.15.43:0/718200618 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f66bc001970 con 0x7f66c4142930 2023-06-28T20:07:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f66c1d9b700 1 -- 172.21.15.43:0/718200618 >> v1:172.21.15.43:6789/0 conn(0x7f66c4142930 legacy=0x7f66c40cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f66c1d9b700 1 --2- 172.21.15.43:0/718200618 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66ac004c20 0x7f66ac005010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f66c1d9b700 1 --2- 172.21.15.43:0/718200618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66ac0037f0 0x7f66ac003c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f66c1d9b700 1 --2- 172.21.15.43:0/718200618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66ac0041a0 0x7f66ac00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f66c1d9b700 1 -- 172.21.15.43:0/718200618 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c4142d00 con 0x7f66ac0037f0 2023-06-28T20:07:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f66c1d9b700 1 -- 172.21.15.43:0/718200618 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac00c0b0 con 0x7f66ac0041a0 2023-06-28T20:07:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f66c1d9b700 1 -- 172.21.15.43:0/718200618 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac00c270 con 0x7f66ac004c20 2023-06-28T20:07:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7fb9d2004700 1 -- 172.21.15.43:0/604936165 shutdown_connections 2023-06-28T20:07:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7fb9d2004700 1 --2- 172.21.15.43:0/604936165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9cc158ef0 0x7fb9cc159370 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7fb9d2004700 1 --2- 172.21.15.43:0/604936165 >> v2:172.21.15.43:3300/0 conn(0x7fb9cc06a9d0 0x7fb9cc06cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7fb9d2004700 1 --2- 172.21.15.43:0/604936165 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9cc0ca480 0x7fb9cc0ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7fb9d2004700 1 -- 172.21.15.43:0/604936165 >> 172.21.15.43:0/604936165 conn(0x7fb9cc069ef0 msgr2=0x7fb9cc06a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f66c2d9d700 1 --2- 172.21.15.43:0/718200618 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66ac004c20 0x7f66ac005010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7ff572ffd700 1 --2- 172.21.15.43:0/2270509348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5740b0130 0x7ff574158b60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009de0 tx=0x7ff56c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3571955712416a44 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f07e57f6700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e0158ef0 0x7f07e006cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f07e57f6700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f07e0159730 0x7f07e00cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f07e57f6700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e006d610 con 0x7f07e01592c0 2023-06-28T20:07:27.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f07e57f6700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e015c110 con 0x7f07e0159730 2023-06-28T20:07:27.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f07e57f6700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e015c3f0 con 0x7f07e0158ef0 2023-06-28T20:07:27.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f66c259c700 1 --2- 172.21.15.43:0/718200618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66ac0037f0 0x7f66ac003c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f07deffd700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e0158ef0 0x7f07e006cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f07deffd700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e0158ef0 0x7f07e006cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.43:37142/0 (socket says 172.21.15.43:37142) 2023-06-28T20:07:27.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f07deffd700 1 -- 172.21.15.43:0/3369735704 learned_addr learned my addr 172.21.15.43:0/3369735704 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f6746a68700 1 -- 172.21.15.43:0/1912407117 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f674006d8b0 msgr2=0x7f6728003a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f6746a68700 1 --2- 172.21.15.43:0/1912407117 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f674006d8b0 0x7f6728003a30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f673000ec20 tx=0x7f6728005880 comp rx=0 tx=0).stop 2023-06-28T20:07:27.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f6745005700 1 -- 172.21.15.43:0/1912407117 reap_dead start 2023-06-28T20:07:27.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f6746a68700 1 -- 172.21.15.43:0/1912407117 shutdown_connections 2023-06-28T20:07:27.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f6746a68700 1 -- 172.21.15.43:0/1912407117 >> 172.21.15.43:0/1912407117 conn(0x7f6740069ef0 msgr2=0x7f674006a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7ff5727fc700 1 -- 172.21.15.43:0/2270509348 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c00e070 con 0x7ff5740b0130 2023-06-28T20:07:27.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f07df7fe700 1 --2- 172.21.15.43:0/3369735704 >> v2:172.21.15.43:3300/0 conn(0x7f07e0159730 0x7f07e00cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f07ddffb700 1 -- 172.21.15.43:0/3369735704 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585108789 0 0) 0x7f07e006d610 con 0x7f07e01592c0 2023-06-28T20:07:27.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f07ddffb700 1 -- 172.21.15.43:0/3369735704 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07c80035b0 con 0x7f07e01592c0 2023-06-28T20:07:27.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7ff5727fc700 1 -- 172.21.15.43:0/2270509348 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56c00bbd0 con 0x7ff5740b0130 2023-06-28T20:07:27.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7ff5727fc700 1 -- 172.21.15.43:0/2270509348 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c012430 con 0x7ff5740b0130 2023-06-28T20:07:27.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f6746a68700 1 -- 172.21.15.43:0/1912407117 shutdown_connections 2023-06-28T20:07:27.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.794+0000 7f5455533700 1 -- start start 2023-06-28T20:07:27.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f07deffd700 1 -- 172.21.15.43:0/3369735704 >> v2:172.21.15.43:3300/0 conn(0x7f07e0159730 msgr2=0x7f07e00cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f07deffd700 1 --2- 172.21.15.43:0/3369735704 >> v2:172.21.15.43:3300/0 conn(0x7f07e0159730 0x7f07e00cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f07deffd700 1 -- 172.21.15.43:0/3369735704 >> v1:172.21.15.178:6789/0 conn(0x7f07e01592c0 legacy=0x7f07e00ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f07deffd700 1 -- 172.21.15.43:0/3369735704 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e015c6d0 con 0x7f07e0158ef0 2023-06-28T20:07:27.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f5455533700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5450158ef0 0x7f54501f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f5455533700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54501592c0 0x7f5450143460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f5455533700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5450159710 0x7f54500ce5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f5455533700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5450154710 con 0x7f5450159710 2023-06-28T20:07:27.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f5455533700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54501439a0 con 0x7f54501592c0 2023-06-28T20:07:27.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f5455533700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54500b45d0 con 0x7f5450158ef0 2023-06-28T20:07:27.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.793+0000 7f17c078a700 1 -- 172.21.15.43:0/1118029552 wait complete. 2023-06-28T20:07:27.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f17c078a700 1 Processor -- start 2023-06-28T20:07:27.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7f544e7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54501592c0 0x7f5450143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7fb9d2004700 1 -- 172.21.15.43:0/604936165 shutdown_connections 2023-06-28T20:07:27.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7f544e7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54501592c0 0x7f5450143460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:39244/0 (socket says 172.21.15.43:39244) 2023-06-28T20:07:27.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7f544e7fc700 1 -- 172.21.15.43:0/3222106812 learned_addr learned my addr 172.21.15.43:0/3222106812 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7f544f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5450159710 0x7f54500ce5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7ff57a0ec700 1 -- 172.21.15.43:0/2270509348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5740b0130 msgr2=0x7ff574158b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7ff57a0ec700 1 --2- 172.21.15.43:0/2270509348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5740b0130 0x7ff574158b60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009de0 tx=0x7ff56c00b330 comp rx=0 tx=0).stop 2023-06-28T20:07:27.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7f07deffd700 1 --2- 172.21.15.43:0/3369735704 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e0158ef0 0x7f07e006cf10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f07d0006fd0 tx=0x7f07d000d330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d09da656f9b7147 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7fb9d2004700 1 -- 172.21.15.43:0/604936165 wait complete. 2023-06-28T20:07:27.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7f544effd700 1 --2- 172.21.15.43:0/3222106812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5450158ef0 0x7f54501f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7ff57a0ec700 1 -- 172.21.15.43:0/2270509348 shutdown_connections 2023-06-28T20:07:27.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7ff57a0ec700 1 --2- 172.21.15.43:0/2270509348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5740b0130 0x7ff574158b60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7ff57a0ec700 1 -- 172.21.15.43:0/2270509348 >> 172.21.15.43:0/2270509348 conn(0x7ff5740cfa80 msgr2=0x7ff5740cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7f544f7fe700 1 -- 172.21.15.43:0/3222106812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5450158ef0 msgr2=0x7f54501f21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7f544f7fe700 1 --2- 172.21.15.43:0/3222106812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5450158ef0 0x7f54501f21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7f544f7fe700 1 -- 172.21.15.43:0/3222106812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54501592c0 msgr2=0x7f5450143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7f544f7fe700 1 --2- 172.21.15.43:0/3222106812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54501592c0 0x7f5450143460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7f544f7fe700 1 -- 172.21.15.43:0/3222106812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54400085d0 con 0x7f5450159710 2023-06-28T20:07:27.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7f544effd700 1 --2- 172.21.15.43:0/3222106812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5450158ef0 0x7f54501f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:27.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7f07ddffb700 1 -- 172.21.15.43:0/3369735704 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d0010070 con 0x7f07e0158ef0 2023-06-28T20:07:27.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7f07ddffb700 1 -- 172.21.15.43:0/3369735704 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d0005a20 con 0x7f07e0158ef0 2023-06-28T20:07:27.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7f07ddffb700 1 -- 172.21.15.43:0/3369735704 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d0005dd0 con 0x7f07e0158ef0 2023-06-28T20:07:27.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f544f7fe700 1 --2- 172.21.15.43:0/3222106812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5450159710 0x7f54500ce5f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f544001a000 tx=0x7f5440004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7f17c078a700 1 -- start start 2023-06-28T20:07:27.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f17c078a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17b808a660 0x7f17b816a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f17c078a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b80cde80 0x7f17b80b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f17c078a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17b80cf0b0 0x7f17b80b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f17c078a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b816c710 con 0x7f17b808a660 2023-06-28T20:07:27.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f17c078a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b80af1f0 con 0x7f17b80cf0b0 2023-06-28T20:07:27.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f17c078a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b80af4d0 con 0x7f17b80cde80 2023-06-28T20:07:27.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f66c2d9d700 1 -- 172.21.15.43:0/718200618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66ac0041a0 msgr2=0x7f66ac00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f66c2d9d700 1 --2- 172.21.15.43:0/718200618 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66ac0041a0 0x7f66ac00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f66c2d9d700 1 -- 172.21.15.43:0/718200618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66ac0037f0 msgr2=0x7f66ac003c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f66c2d9d700 1 --2- 172.21.15.43:0/718200618 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66ac0037f0 0x7f66ac003c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f66c2d9d700 1 -- 172.21.15.43:0/718200618 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c406cdb0 con 0x7f66ac004c20 2023-06-28T20:07:27.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.796+0000 7f66c2d9d700 1 --2- 172.21.15.43:0/718200618 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66ac004c20 0x7f66ac005010 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00a7e0 tx=0x7f66ac005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db82dbbbea4829ff server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.795+0000 7f6746a68700 1 -- 172.21.15.43:0/1912407117 wait complete. 2023-06-28T20:07:27.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f6746a68700 1 Processor -- start 2023-06-28T20:07:27.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f17be526700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17b808a660 0x7f17b816a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f17be526700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17b808a660 0x7f17b816a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53182/0 (socket says 172.21.15.43:53182) 2023-06-28T20:07:27.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f17bed27700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17b80cf0b0 0x7f17b80b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f17be526700 1 -- 172.21.15.43:0/3084930751 learned_addr learned my addr 172.21.15.43:0/3084930751 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7ff57a0ec700 1 -- 172.21.15.43:0/2270509348 shutdown_connections 2023-06-28T20:07:27.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7ff57a0ec700 1 -- 172.21.15.43:0/2270509348 wait complete. 2023-06-28T20:07:27.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f17be526700 1 -- 172.21.15.43:0/3084930751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b80cde80 msgr2=0x7f17b80b3300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-28T20:07:27.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f17be526700 1 --2- 172.21.15.43:0/3084930751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b80cde80 0x7f17b80b3300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f17be526700 1 -- 172.21.15.43:0/3084930751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17b80cf0b0 msgr2=0x7f17b80b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f17be526700 1 --2- 172.21.15.43:0/3084930751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17b80cf0b0 0x7f17b80b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f17be526700 1 -- 172.21.15.43:0/3084930751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b0014660 con 0x7f17b808a660 2023-06-28T20:07:27.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.798+0000 7fb9d2004700 1 Processor -- start 2023-06-28T20:07:27.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f07e57f6700 1 -- 172.21.15.43:0/3369735704 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e0158ef0 msgr2=0x7f07e006cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f07e57f6700 1 --2- 172.21.15.43:0/3369735704 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e0158ef0 0x7f07e006cf10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f07d0006fd0 tx=0x7f07d000d330 comp rx=0 tx=0).stop 2023-06-28T20:07:27.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.798+0000 7fb9d2004700 1 -- start start 2023-06-28T20:07:27.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f54377fe700 1 -- 172.21.15.43:0/3222106812 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544001f070 con 0x7f5450159710 2023-06-28T20:07:27.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f54377fe700 1 -- 172.21.15.43:0/3222106812 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5440020040 con 0x7f5450159710 2023-06-28T20:07:27.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f54377fe700 1 -- 172.21.15.43:0/3222106812 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5440004470 con 0x7f5450159710 2023-06-28T20:07:27.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.798+0000 7f5455533700 1 -- 172.21.15.43:0/3222106812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54500ce030 con 0x7f5450159710 2023-06-28T20:07:27.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.798+0000 7f5455533700 1 -- 172.21.15.43:0/3222106812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54500b48b0 con 0x7f5450159710 2023-06-28T20:07:27.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f66c1d9b700 1 -- 172.21.15.43:0/718200618 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc00a250 con 0x7f66ac004c20 2023-06-28T20:07:27.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f66c1d9b700 1 -- 172.21.15.43:0/718200618 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66bc012660 con 0x7f66ac004c20 2023-06-28T20:07:27.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.797+0000 7f66c1d9b700 1 -- 172.21.15.43:0/718200618 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc012a10 con 0x7f66ac004c20 2023-06-28T20:07:27.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.798+0000 7f66c9145700 1 -- 172.21.15.43:0/718200618 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66ac004c20 msgr2=0x7f66ac005010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.798+0000 7f66c9145700 1 --2- 172.21.15.43:0/718200618 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66ac004c20 0x7f66ac005010 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00a7e0 tx=0x7f66ac005880 comp rx=0 tx=0).stop 2023-06-28T20:07:27.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.798+0000 7f66c359e700 1 -- 172.21.15.43:0/718200618 reap_dead start 2023-06-28T20:07:27.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.798+0000 7f66c9145700 1 -- 172.21.15.43:0/718200618 shutdown_connections 2023-06-28T20:07:27.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.798+0000 7f66c9145700 1 -- 172.21.15.43:0/718200618 >> 172.21.15.43:0/718200618 conn(0x7f66c4069ef0 msgr2=0x7f66c406a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.798+0000 7f66c9145700 1 -- 172.21.15.43:0/718200618 shutdown_connections 2023-06-28T20:07:27.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.799+0000 7f6746a68700 1 -- start start 2023-06-28T20:07:27.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.799+0000 7ff57a0ec700 1 Processor -- start 2023-06-28T20:07:27.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.799+0000 7f07e57f6700 1 -- 172.21.15.43:0/3369735704 shutdown_connections 2023-06-28T20:07:27.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.799+0000 7f07e57f6700 1 --2- 172.21.15.43:0/3369735704 >> v2:172.21.15.43:3300/0 conn(0x7f07e0159730 0x7f07e00cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.799+0000 7f07e57f6700 1 --2- 172.21.15.43:0/3369735704 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e0158ef0 0x7f07e006cf10 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.799+0000 7f07e57f6700 1 -- 172.21.15.43:0/3369735704 >> 172.21.15.43:0/3369735704 conn(0x7f07e0069ef0 msgr2=0x7f07e006a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.799+0000 7fb9d2004700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9cc06a9d0 0x7fb9cc1576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.799+0000 7fb9d2004700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9cc0ca480 0x7fb9cc144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.799+0000 7fb9d2004700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9cc158ef0 0x7fb9cc146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.799+0000 7fb9d2004700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc0cdf10 con 0x7fb9cc0ca480 2023-06-28T20:07:27.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.799+0000 7fb9d2004700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc13c000 con 0x7fb9cc06a9d0 2023-06-28T20:07:27.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.799+0000 7fb9d2004700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc144170 con 0x7fb9cc158ef0 2023-06-28T20:07:27.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.799+0000 7fb9cb7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9cc06a9d0 0x7fb9cc1576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.799+0000 7fb9cb7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9cc06a9d0 0x7fb9cc1576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:39252/0 (socket says 172.21.15.43:39252) 2023-06-28T20:07:27.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.799+0000 7fb9cb7fe700 1 -- 172.21.15.43:0/180874331 learned_addr learned my addr 172.21.15.43:0/180874331 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.799+0000 7fb9cbfff700 1 --2- 172.21.15.43:0/180874331 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9cc158ef0 0x7fb9cc146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.799+0000 7fb9caffd700 1 --2- 172.21.15.43:0/180874331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9cc0ca480 0x7fb9cc144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.800+0000 7f6746a68700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f674006d8b0 0x7f67401fe4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.800+0000 7f6746a68700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67400cd5e0 0x7f67400cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.800+0000 7f6746a68700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67400b3620 0x7f67400b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.800+0000 7f6746a68700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6740202db0 con 0x7f67400b3620 2023-06-28T20:07:27.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.800+0000 7f6746a68700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67400cdf90 con 0x7f674006d8b0 2023-06-28T20:07:27.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.800+0000 7f6746a68700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67401fea00 con 0x7f67400cd5e0 2023-06-28T20:07:27.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.800+0000 7f5455533700 1 -- 172.21.15.43:0/3222106812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5450154850 con 0x7f5450159710 2023-06-28T20:07:27.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.800+0000 7fb9cb7fe700 1 -- 172.21.15.43:0/180874331 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9cc158ef0 msgr2=0x7fb9cc146710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.800+0000 7fb9cb7fe700 1 --2- 172.21.15.43:0/180874331 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9cc158ef0 0x7fb9cc146710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.800+0000 7fb9cb7fe700 1 -- 172.21.15.43:0/180874331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9cc0ca480 msgr2=0x7fb9cc144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.800+0000 7fb9cb7fe700 1 --2- 172.21.15.43:0/180874331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9cc0ca480 0x7fb9cc144600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.800+0000 7fb9cb7fe700 1 -- 172.21.15.43:0/180874331 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c0009500 con 0x7fb9cc06a9d0 2023-06-28T20:07:27.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.800+0000 7fb9cb7fe700 1 --2- 172.21.15.43:0/180874331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9cc06a9d0 0x7fb9cc1576e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000e9d0 tx=0x7fb9c000ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.800+0000 7f07e57f6700 1 -- 172.21.15.43:0/3369735704 shutdown_connections 2023-06-28T20:07:27.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.800+0000 7ff57a0ec700 1 -- start start 2023-06-28T20:07:27.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.800+0000 7f07e57f6700 1 -- 172.21.15.43:0/3369735704 wait complete. 2023-06-28T20:07:27.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.801+0000 7fb9b3fff700 1 -- 172.21.15.43:0/180874331 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0005a20 con 0x7fb9cc06a9d0 2023-06-28T20:07:27.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7f6744804700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f674006d8b0 0x7f67401fe4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7f673ffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67400cd5e0 0x7f67400cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7f6744804700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f674006d8b0 0x7f67401fe4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:39264/0 (socket says 172.21.15.43:39264) 2023-06-28T20:07:27.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7f6744804700 1 -- 172.21.15.43:0/3263364150 learned_addr learned my addr 172.21.15.43:0/3263364150 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.801+0000 7f54377fe700 1 -- 172.21.15.43:0/3222106812 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f54400036a0 con 0x7f5450159710 2023-06-28T20:07:27.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.801+0000 7fb9b3fff700 1 -- 172.21.15.43:0/180874331 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c0005b80 con 0x7fb9cc06a9d0 2023-06-28T20:07:27.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.801+0000 7fb9b3fff700 1 -- 172.21.15.43:0/180874331 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c000c850 con 0x7fb9cc06a9d0 2023-06-28T20:07:27.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7fb9d2004700 1 -- 172.21.15.43:0/180874331 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9bc0094a0 con 0x7fb9cc06a9d0 2023-06-28T20:07:27.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7fb9d2004700 1 -- 172.21.15.43:0/180874331 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9cc143b20 con 0x7fb9cc06a9d0 2023-06-28T20:07:27.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.799+0000 7f66c9145700 1 -- 172.21.15.43:0/718200618 wait complete. 2023-06-28T20:07:27.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7f66c9145700 1 Processor -- start 2023-06-28T20:07:27.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.801+0000 7ff57a0ec700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff574066e90 0x7ff5740b3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.801+0000 7ff57a0ec700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5740b0130 0x7ff5740b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.801+0000 7ff57a0ec700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5740c9400 0x7ff5740ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.801+0000 7ff57a0ec700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740cb6a0 con 0x7ff5740b0130 2023-06-28T20:07:27.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.801+0000 7ff57a0ec700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5741488f0 con 0x7ff574066e90 2023-06-28T20:07:27.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.801+0000 7ff57a0ec700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740cb3c0 con 0x7ff5740c9400 2023-06-28T20:07:27.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.801+0000 7ff5737fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff574066e90 0x7ff5740b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7ff5737fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff574066e90 0x7ff5740b3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:39258/0 (socket says 172.21.15.43:39258) 2023-06-28T20:07:27.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7ff5737fe700 1 -- 172.21.15.43:0/2693296331 learned_addr learned my addr 172.21.15.43:0/2693296331 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7ff573fff700 1 --2- 172.21.15.43:0/2693296331 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5740c9400 0x7ff5740ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7ff572ffd700 1 --2- 172.21.15.43:0/2693296331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5740b0130 0x7ff5740b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.800+0000 7f17be526700 1 --2- 172.21.15.43:0/3084930751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17b808a660 0x7f17b816a7f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f17b40209d0 tx=0x7f17b4020d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.801+0000 7f54377fe700 1 --2- 172.21.15.43:0/3222106812 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f543c06adb0 0x7f543c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.801+0000 7f54377fe700 1 -- 172.21.15.43:0/3222106812 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f544008ef10 con 0x7f5450159710 2023-06-28T20:07:27.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7ff573fff700 1 -- 172.21.15.43:0/2693296331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff574066e90 msgr2=0x7ff5740b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7ff573fff700 1 --2- 172.21.15.43:0/2693296331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff574066e90 0x7ff5740b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7ff573fff700 1 -- 172.21.15.43:0/2693296331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5740b0130 msgr2=0x7ff5740b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7ff573fff700 1 --2- 172.21.15.43:0/2693296331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5740b0130 0x7ff5740b3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7ff573fff700 1 -- 172.21.15.43:0/2693296331 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56c009a70 con 0x7ff5740c9400 2023-06-28T20:07:27.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7ff572ffd700 1 --2- 172.21.15.43:0/2693296331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5740b0130 0x7ff5740b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:27.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7ff5737fe700 1 --2- 172.21.15.43:0/2693296331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff574066e90 0x7ff5740b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:27.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7f673ffff700 1 -- 172.21.15.43:0/3263364150 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f674006d8b0 msgr2=0x7f67401fe4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7f673ffff700 1 --2- 172.21.15.43:0/3263364150 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f674006d8b0 0x7f67401fe4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7f673ffff700 1 -- 172.21.15.43:0/3263364150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67400b3620 msgr2=0x7f67400b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7f673ffff700 1 --2- 172.21.15.43:0/3263364150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67400b3620 0x7f67400b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7f673ffff700 1 -- 172.21.15.43:0/3263364150 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6740202130 con 0x7f67400cd5e0 2023-06-28T20:07:27.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.803+0000 7ff573fff700 1 --2- 172.21.15.43:0/2693296331 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5740c9400 0x7ff5740ca870 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff568000f60 tx=0x7ff56800c670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.803+0000 7f07e57f6700 1 Processor -- start 2023-06-28T20:07:27.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.803+0000 7f673ffff700 1 --2- 172.21.15.43:0/3263364150 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67400cd5e0 0x7f67400cda50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f673800e4c0 tx=0x7f673800e880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.803+0000 7f673d7fa700 1 -- 172.21.15.43:0/3263364150 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6738012590 con 0x7f67400cd5e0 2023-06-28T20:07:27.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.803+0000 7f673d7fa700 1 -- 172.21.15.43:0/3263364150 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67380126f0 con 0x7f67400cd5e0 2023-06-28T20:07:27.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.803+0000 7f673d7fa700 1 -- 172.21.15.43:0/3263364150 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6738012a80 con 0x7f67400cd5e0 2023-06-28T20:07:27.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.802+0000 7f66c9145700 1 -- start start 2023-06-28T20:07:27.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.803+0000 7f66c9145700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c4202a90 0x7f66c41441d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.803+0000 7f66c9145700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66c4202e60 0x7f66c40b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.803+0000 7f66c9145700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66c40b1310 0x7f66c41fe740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.803+0000 7f66c9145700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c40b43e0 con 0x7f66c4202a90 2023-06-28T20:07:27.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.803+0000 7f66c9145700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c40b46c0 con 0x7f66c4202e60 2023-06-28T20:07:27.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.803+0000 7f66c9145700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c41439d0 con 0x7f66c40b1310 2023-06-28T20:07:27.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.803+0000 7f66c259c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66c4202e60 0x7f66c40b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.803+0000 7f66c259c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66c4202e60 0x7f66c40b0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:39272/0 (socket says 172.21.15.43:39272) 2023-06-28T20:07:27.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.803+0000 7f66c259c700 1 -- 172.21.15.43:0/1688110000 learned_addr learned my addr 172.21.15.43:0/1688110000 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.803+0000 7f66c2d9d700 1 --2- 172.21.15.43:0/1688110000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c4202a90 0x7f66c41441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.803+0000 7ff55bfff700 1 -- 172.21.15.43:0/2693296331 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5680052d0 con 0x7ff5740c9400 2023-06-28T20:07:27.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.803+0000 7ff55bfff700 1 -- 172.21.15.43:0/2693296331 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff568005430 con 0x7ff5740c9400 2023-06-28T20:07:27.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.803+0000 7ff55bfff700 1 -- 172.21.15.43:0/2693296331 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5680038c0 con 0x7ff5740c9400 2023-06-28T20:07:27.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.803+0000 7ff57a0ec700 1 -- 172.21.15.43:0/2693296331 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff574134170 con 0x7ff5740c9400 2023-06-28T20:07:27.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.804+0000 7ff57a0ec700 1 -- 172.21.15.43:0/2693296331 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff574134730 con 0x7ff5740c9400 2023-06-28T20:07:27.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.804+0000 7f66c359e700 1 --2- 172.21.15.43:0/1688110000 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66c40b1310 0x7f66c41fe740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.804+0000 7f66c259c700 1 -- 172.21.15.43:0/1688110000 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66c40b1310 msgr2=0x7f66c41fe740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.804+0000 7f66c259c700 1 --2- 172.21.15.43:0/1688110000 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66c40b1310 0x7f66c41fe740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.804+0000 7f66c259c700 1 -- 172.21.15.43:0/1688110000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c4202a90 msgr2=0x7f66c41441d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.804+0000 7f66c259c700 1 --2- 172.21.15.43:0/1688110000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c4202a90 0x7f66c41441d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.804+0000 7f66c259c700 1 -- 172.21.15.43:0/1688110000 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c4143cb0 con 0x7f66c4202e60 2023-06-28T20:07:27.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.804+0000 7f07e57f6700 1 -- start start 2023-06-28T20:07:27.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.805+0000 7f6746a68700 1 -- 172.21.15.43:0/3263364150 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6740202410 con 0x7f67400cd5e0 2023-06-28T20:07:27.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.805+0000 7f6746a68700 1 -- 172.21.15.43:0/3263364150 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67402029d0 con 0x7f67400cd5e0 2023-06-28T20:07:27.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.805+0000 7ff57a0ec700 1 -- 172.21.15.43:0/2693296331 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5740728b0 con 0x7ff5740c9400 2023-06-28T20:07:27.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.805+0000 7f17aeffd700 1 -- 172.21.15.43:0/3084930751 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b40096f0 con 0x7f17b808a660 2023-06-28T20:07:27.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.805+0000 7f544effd700 1 --2- 172.21.15.43:0/3222106812 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f543c06adb0 0x7f543c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.805+0000 7f17aeffd700 1 -- 172.21.15.43:0/3084930751 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b401ea70 con 0x7f17b808a660 2023-06-28T20:07:27.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.805+0000 7f17aeffd700 1 -- 172.21.15.43:0/3084930751 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b401ede0 con 0x7f17b808a660 2023-06-28T20:07:27.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.805+0000 7f17c078a700 1 -- 172.21.15.43:0/3084930751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b8169d10 con 0x7f17b808a660 2023-06-28T20:07:27.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.805+0000 7f17c078a700 1 -- 172.21.15.43:0/3084930751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b816a2d0 con 0x7f17b808a660 2023-06-28T20:07:27.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.805+0000 7fb9b3fff700 1 -- 172.21.15.43:0/180874331 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb9c000c9b0 con 0x7fb9cc06a9d0 2023-06-28T20:07:27.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.805+0000 7fb9b3fff700 1 --2- 172.21.15.43:0/180874331 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb9b806adb0 0x7fb9b806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.806+0000 7fb9b3fff700 1 -- 172.21.15.43:0/180874331 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb9c00896c0 con 0x7fb9cc06a9d0 2023-06-28T20:07:27.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.806+0000 7f544effd700 1 --2- 172.21.15.43:0/3222106812 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f543c06adb0 0x7f543c06d260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f545015bb20 tx=0x7f5444008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.806+0000 7ff55bfff700 1 -- 172.21.15.43:0/2693296331 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff568003a20 con 0x7ff5740c9400 2023-06-28T20:07:27.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.806+0000 7fb9caffd700 1 --2- 172.21.15.43:0/180874331 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb9b806adb0 0x7fb9b806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.806+0000 7f66c259c700 1 --2- 172.21.15.43:0/1688110000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66c4202e60 0x7f66c40b0dd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f66b4001670 tx=0x7f66b4001980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.806+0000 7f6746a68700 1 -- 172.21.15.43:0/3263364150 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6740202ef0 con 0x7f67400cd5e0 2023-06-28T20:07:27.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.806+0000 7f17c078a700 1 -- 172.21.15.43:0/3084930751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b816a4f0 con 0x7f17b808a660 2023-06-28T20:07:27.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.806+0000 7f66ab7fe700 1 -- 172.21.15.43:0/1688110000 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b40114f0 con 0x7f66c4202e60 2023-06-28T20:07:27.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.806+0000 7f07e57f6700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e0158ef0 0x7f07e01554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.806+0000 7f07e57f6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07e01592c0 0x7f07e0202860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.806+0000 7f07e57f6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07e0159730 0x7f07e00ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.806+0000 7f07e57f6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e01ffa70 con 0x7f07e0159730 2023-06-28T20:07:27.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.806+0000 7f07e57f6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e01feb30 con 0x7f07e01592c0 2023-06-28T20:07:27.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.806+0000 7f07e57f6700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e01fedf0 con 0x7f07e0158ef0 2023-06-28T20:07:27.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.807+0000 7fb9caffd700 1 --2- 172.21.15.43:0/180874331 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb9b806adb0 0x7fb9b806d260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc06d180 tx=0x7fb9c400d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.806+0000 7f66ab7fe700 1 -- 172.21.15.43:0/1688110000 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b4011650 con 0x7f66c4202e60 2023-06-28T20:07:27.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.806+0000 7f66ab7fe700 1 -- 172.21.15.43:0/1688110000 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b4011a00 con 0x7f66c4202e60 2023-06-28T20:07:27.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.807+0000 7f66c9145700 1 -- 172.21.15.43:0/1688110000 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c42035f0 con 0x7f66c4202e60 2023-06-28T20:07:27.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.807+0000 7f66c9145700 1 -- 172.21.15.43:0/1688110000 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c41fec80 con 0x7f66c4202e60 2023-06-28T20:07:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.807+0000 7f07de7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07e01592c0 0x7f07e0202860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.807+0000 7f07de7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07e01592c0 0x7f07e0202860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:39280/0 (socket says 172.21.15.43:39280) 2023-06-28T20:07:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.807+0000 7f07de7fc700 1 -- 172.21.15.43:0/94038013 learned_addr learned my addr 172.21.15.43:0/94038013 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.807+0000 7f673d7fa700 1 -- 172.21.15.43:0/3263364150 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6738012be0 con 0x7f67400cd5e0 2023-06-28T20:07:27.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.807+0000 7ff55bfff700 1 --2- 172.21.15.43:0/2693296331 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff56006adb0 0x7ff56006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.808+0000 7ff55bfff700 1 -- 172.21.15.43:0/2693296331 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff5680886d0 con 0x7ff5740c9400 2023-06-28T20:07:27.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.808+0000 7ff5737fe700 1 --2- 172.21.15.43:0/2693296331 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff56006adb0 0x7ff56006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.807+0000 7f07df7fe700 1 --2- 172.21.15.43:0/94038013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07e0159730 0x7f07e00ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.807+0000 7f07df7fe700 1 -- 172.21.15.43:0/94038013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e0158ef0 msgr2=0x7f07e01554e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-28T20:07:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.807+0000 7f07df7fe700 1 --2- 172.21.15.43:0/94038013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e0158ef0 0x7f07e01554e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.807+0000 7f07df7fe700 1 -- 172.21.15.43:0/94038013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07e01592c0 msgr2=0x7f07e0202860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.807+0000 7f07df7fe700 1 --2- 172.21.15.43:0/94038013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07e01592c0 0x7f07e0202860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.807+0000 7f07df7fe700 1 -- 172.21.15.43:0/94038013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d0008580 con 0x7f07e0159730 2023-06-28T20:07:27.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.807+0000 7f07de7fc700 1 --2- 172.21.15.43:0/94038013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07e01592c0 0x7f07e0202860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:27.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.807+0000 7f07df7fe700 1 --2- 172.21.15.43:0/94038013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07e0159730 0x7f07e00ca480 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f07d800eb70 tx=0x7f07d800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.808+0000 7f673d7fa700 1 --2- 172.21.15.43:0/3263364150 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f672c06ace0 0x7f672c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.808+0000 7f673d7fa700 1 -- 172.21.15.43:0/3263364150 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6738089fd0 con 0x7f67400cd5e0 2023-06-28T20:07:27.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.808+0000 7f6744804700 1 --2- 172.21.15.43:0/3263364150 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f672c06ace0 0x7f672c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.808+0000 7ff5737fe700 1 --2- 172.21.15.43:0/2693296331 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff56006adb0 0x7ff56006d260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff564005910 tx=0x7ff564009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.808+0000 7f6744804700 1 --2- 172.21.15.43:0/3263364150 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f672c06ace0 0x7f672c06d190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f673000e350 tx=0x7f6730014000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.809+0000 7f66c9145700 1 -- 172.21.15.43:0/1688110000 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c406bae0 con 0x7f66c4202e60 2023-06-28T20:07:27.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.809+0000 7f66ab7fe700 1 -- 172.21.15.43:0/1688110000 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f66b4011b60 con 0x7f66c4202e60 2023-06-28T20:07:27.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.809+0000 7f54377fe700 1 -- 172.21.15.43:0/3222106812 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f544005a150 con 0x7f5450159710 2023-06-28T20:07:27.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.809+0000 7f66ab7fe700 1 --2- 172.21.15.43:0/1688110000 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f66b006ad30 0x7f66b006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.810+0000 7f66c2d9d700 1 --2- 172.21.15.43:0/1688110000 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f66b006ad30 0x7f66b006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.810+0000 7f66ab7fe700 1 -- 172.21.15.43:0/1688110000 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f66b4088d20 con 0x7f66c4202e60 2023-06-28T20:07:27.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.810+0000 7f66c2d9d700 1 --2- 172.21.15.43:0/1688110000 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f66b006ad30 0x7f66b006d1e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f66bc000f80 tx=0x7f66bc005040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.810+0000 7f17aeffd700 1 -- 172.21.15.43:0/3084930751 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f17b4039480 con 0x7f17b808a660 2023-06-28T20:07:27.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.810+0000 7f07c77fe700 1 -- 172.21.15.43:0/94038013 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d800ca00 con 0x7f07e0159730 2023-06-28T20:07:27.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.810+0000 7f07c77fe700 1 -- 172.21.15.43:0/94038013 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d800f040 con 0x7f07e0159730 2023-06-28T20:07:27.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.811+0000 7f07c77fe700 1 -- 172.21.15.43:0/94038013 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d800cd80 con 0x7f07e0159730 2023-06-28T20:07:27.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.811+0000 7fb9d2004700 1 -- 172.21.15.43:0/180874331 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9cc144450 con 0x7fb9cc06a9d0 2023-06-28T20:07:27.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.811+0000 7f07e57f6700 1 -- 172.21.15.43:0/94038013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e01ff0d0 con 0x7f07e0159730 2023-06-28T20:07:27.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.811+0000 7f17aeffd700 1 --2- 172.21.15.43:0/3084930751 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f179c06ace0 0x7f179c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.812+0000 7fba7cc1d700 1 Processor -- start 2023-06-28T20:07:27.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.812+0000 7fba7cc1d700 1 -- start start 2023-06-28T20:07:27.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.811+0000 7f07e57f6700 1 -- 172.21.15.43:0/94038013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e01ff690 con 0x7f07e0159730 2023-06-28T20:07:27.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.811+0000 7f17aeffd700 1 -- 172.21.15.43:0/3084930751 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f17b409b110 con 0x7f17b808a660 2023-06-28T20:07:27.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.811+0000 7f17bdd25700 1 --2- 172.21.15.43:0/3084930751 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f179c06ace0 0x7f179c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.812+0000 7f07e57f6700 1 -- 172.21.15.43:0/94038013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c00052f0 con 0x7f07e0159730 2023-06-28T20:07:27.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.812+0000 7f17bdd25700 1 --2- 172.21.15.43:0/3084930751 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f179c06ace0 0x7f179c06d190 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f17b8164850 tx=0x7f17a8007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.813+0000 7fba7cc1d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fba780ca850 0x7fba7815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.813+0000 7fba7cc1d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba780cbac0 0x7fba7806d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.813+0000 7fba7cc1d700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba7815df80 con 0x7fba7806cd20 2023-06-28T20:07:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.813+0000 7fba7cc1d700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba7806d6d0 con 0x7fba780ca850 2023-06-28T20:07:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.813+0000 7fba7cc1d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7815aef0 con 0x7fba780cbac0 2023-06-28T20:07:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.813+0000 7fba7659c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fba780ca850 0x7fba7815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.813+0000 7fba7659c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fba780ca850 0x7fba7815a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53230/0 (socket says 172.21.15.43:53230) 2023-06-28T20:07:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.813+0000 7fba7659c700 1 -- 172.21.15.43:0/3278986023 learned_addr learned my addr 172.21.15.43:0/3278986023 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.813+0000 7fba76d9d700 1 --2- 172.21.15.43:0/3278986023 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba780cbac0 0x7fba7806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.813+0000 7fba7659c700 1 -- 172.21.15.43:0/3278986023 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba780cbac0 msgr2=0x7fba7806d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:27.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.813+0000 7fba7659c700 1 --2- 172.21.15.43:0/3278986023 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba780cbac0 0x7fba7806d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.813+0000 7fba7659c700 1 -- 172.21.15.43:0/3278986023 >> v1:172.21.15.178:6789/0 conn(0x7fba7806cd20 legacy=0x7fba780cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.813+0000 7fba7659c700 1 -- 172.21.15.43:0/3278986023 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7815e1d0 con 0x7fba780ca850 2023-06-28T20:07:27.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.813+0000 7fba76d9d700 1 --2- 172.21.15.43:0/3278986023 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba780cbac0 0x7fba7806d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:27.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.813+0000 7fba7659c700 1 --2- 172.21.15.43:0/3278986023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba780ca850 0x7fba7815a9b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fba68006fd0 tx=0x7fba68009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23de678cad477c34 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.815+0000 7ff55bfff700 1 -- 172.21.15.43:0/2693296331 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff5680537f0 con 0x7ff5740c9400 2023-06-28T20:07:27.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.817+0000 7f17aeffd700 1 -- 172.21.15.43:0/3084930751 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f17b40661a0 con 0x7f17b808a660 2023-06-28T20:07:27.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.817+0000 7fba7559a700 1 -- 172.21.15.43:0/3278986023 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba68005e30 con 0x7fba780ca850 2023-06-28T20:07:27.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.818+0000 7f07c77fe700 1 -- 172.21.15.43:0/94038013 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f07d8014480 con 0x7f07e0159730 2023-06-28T20:07:27.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.819+0000 7f66ab7fe700 1 -- 172.21.15.43:0/1688110000 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f66b4053eb0 con 0x7f66c4202e60 2023-06-28T20:07:27.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.819+0000 7f07c77fe700 1 --2- 172.21.15.43:0/94038013 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f07c806b160 0x7f07c806d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.819+0000 7f07c77fe700 1 -- 172.21.15.43:0/94038013 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f07d808a310 con 0x7f07e0159730 2023-06-28T20:07:27.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.819+0000 7f07deffd700 1 --2- 172.21.15.43:0/94038013 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f07c806b160 0x7f07c806d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.820+0000 7f07deffd700 1 --2- 172.21.15.43:0/94038013 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f07c806b160 0x7f07c806d610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f07d000d370 tx=0x7f07d0011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.823+0000 7fba7559a700 1 -- 172.21.15.43:0/3278986023 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba68004030 con 0x7fba780ca850 2023-06-28T20:07:27.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.823+0000 7fba7559a700 1 -- 172.21.15.43:0/3278986023 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba680043e0 con 0x7fba780ca850 2023-06-28T20:07:27.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.824+0000 7fba7cc1d700 1 -- 172.21.15.43:0/3278986023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba780ca850 msgr2=0x7fba7815a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.824+0000 7fba7cc1d700 1 --2- 172.21.15.43:0/3278986023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba780ca850 0x7fba7815a9b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fba68006fd0 tx=0x7fba68009bd0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.824+0000 7f673d7fa700 1 -- 172.21.15.43:0/3263364150 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6740202ef0 con 0x7f67400cd5e0 2023-06-28T20:07:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.826+0000 7fb9b3fff700 1 -- 172.21.15.43:0/180874331 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb9c0054750 con 0x7fb9cc06a9d0 2023-06-28T20:07:27.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.827+0000 7f07c77fe700 1 -- 172.21.15.43:0/94038013 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f07d80554a0 con 0x7f07e0159730 2023-06-28T20:07:27.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.826+0000 7fba7cc1d700 1 -- 172.21.15.43:0/3278986023 shutdown_connections 2023-06-28T20:07:27.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.826+0000 7fba7cc1d700 1 --2- 172.21.15.43:0/3278986023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba780ca850 0x7fba7815a9b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.826+0000 7fba7cc1d700 1 --2- 172.21.15.43:0/3278986023 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba780cbac0 0x7fba7806d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.826+0000 7fba7cc1d700 1 -- 172.21.15.43:0/3278986023 >> 172.21.15.43:0/3278986023 conn(0x7fba78069ef0 msgr2=0x7fba7806a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:27.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.835+0000 7fba7cc1d700 1 -- 172.21.15.43:0/3278986023 shutdown_connections 2023-06-28T20:07:27.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.835+0000 7fba7cc1d700 1 -- 172.21.15.43:0/3278986023 wait complete. 2023-06-28T20:07:27.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.837+0000 7fba7cc1d700 1 Processor -- start 2023-06-28T20:07:27.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.837+0000 7fba7cc1d700 1 -- start start 2023-06-28T20:07:27.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.839+0000 7fba7cc1d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba7806cd20 0x7fba781421e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.839+0000 7fba7cc1d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba780ca850 0x7fba780b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.839+0000 7fba7cc1d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba780cbac0 0x7fba780b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.839+0000 7fba7cc1d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba78140dc0 con 0x7fba780ca850 2023-06-28T20:07:27.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.839+0000 7fba7cc1d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba78142dc0 con 0x7fba7806cd20 2023-06-28T20:07:27.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.839+0000 7fba7cc1d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba781fa580 con 0x7fba780cbac0 2023-06-28T20:07:27.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.839+0000 7fba7659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba7806cd20 0x7fba781421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.839+0000 7fba7659c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba7806cd20 0x7fba781421e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:39302/0 (socket says 172.21.15.43:39302) 2023-06-28T20:07:27.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.839+0000 7fba7659c700 1 -- 172.21.15.43:0/2418895424 learned_addr learned my addr 172.21.15.43:0/2418895424 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:27.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.839+0000 7fba76d9d700 1 --2- 172.21.15.43:0/2418895424 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba780cbac0 0x7fba780b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.840+0000 7fba75d9b700 1 --2- 172.21.15.43:0/2418895424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba780ca850 0x7fba780b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.840+0000 7fba7659c700 1 -- 172.21.15.43:0/2418895424 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba780cbac0 msgr2=0x7fba780b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.840+0000 7fba7659c700 1 --2- 172.21.15.43:0/2418895424 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba780cbac0 0x7fba780b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.840+0000 7fba7659c700 1 -- 172.21.15.43:0/2418895424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba780ca850 msgr2=0x7fba780b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:27.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.840+0000 7fba7659c700 1 --2- 172.21.15.43:0/2418895424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba780ca850 0x7fba780b2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:27.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.840+0000 7fba7659c700 1 -- 172.21.15.43:0/2418895424 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba68009500 con 0x7fba7806cd20 2023-06-28T20:07:27.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.840+0000 7fba7659c700 1 --2- 172.21.15.43:0/2418895424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba7806cd20 0x7fba781421e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fba7806da90 tx=0x7fba68004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:27.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.840+0000 7fba66ffd700 1 -- 172.21.15.43:0/2418895424 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba680045d0 con 0x7fba7806cd20 2023-06-28T20:07:27.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.841+0000 7fba66ffd700 1 -- 172.21.15.43:0/2418895424 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6801d430 con 0x7fba7806cd20 2023-06-28T20:07:27.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.841+0000 7fba66ffd700 1 -- 172.21.15.43:0/2418895424 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6801d7c0 con 0x7fba7806cd20 2023-06-28T20:07:27.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.841+0000 7fba7cc1d700 1 -- 172.21.15.43:0/2418895424 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba78142720 con 0x7fba7806cd20 2023-06-28T20:07:27.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.841+0000 7fba7cc1d700 1 -- 172.21.15.43:0/2418895424 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba781faa10 con 0x7fba7806cd20 2023-06-28T20:07:27.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.843+0000 7fba66ffd700 1 -- 172.21.15.43:0/2418895424 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fba6801d920 con 0x7fba7806cd20 2023-06-28T20:07:27.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.844+0000 7fba66ffd700 1 --2- 172.21.15.43:0/2418895424 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fba6006aec0 0x7fba6006d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:27.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.844+0000 7fba66ffd700 1 -- 172.21.15.43:0/2418895424 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fba6808abf0 con 0x7fba7806cd20 2023-06-28T20:07:27.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.845+0000 7fba7cc1d700 1 -- 172.21.15.43:0/2418895424 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba58005320 con 0x7fba7806cd20 2023-06-28T20:07:27.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.845+0000 7fba75d9b700 1 --2- 172.21.15.43:0/2418895424 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fba6006aec0 0x7fba6006d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:27.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.855+0000 7fba66ffd700 1 -- 172.21.15.43:0/2418895424 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fba68055d10 con 0x7fba7806cd20 2023-06-28T20:07:27.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:27.855+0000 7fba75d9b700 1 --2- 172.21.15.43:0/2418895424 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fba6006aec0 0x7fba6006d370 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fba780ca550 tx=0x7fba6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:28.225 INFO:teuthology.orchestra.run.smithi043.stdout:55834574854 2023-06-28T20:07:28.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.221+0000 7f17c078a700 1 -- 172.21.15.43:0/3084930751 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f17b80728b0 con 0x7f17b808a660 2023-06-28T20:07:28.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.220+0000 7f5455533700 1 -- 172.21.15.43:0/3222106812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f54500728b0 con 0x7f5450159710 2023-06-28T20:07:28.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.221+0000 7f54377fe700 1 -- 172.21.15.43:0/3222106812 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5440006d00 con 0x7f5450159710 2023-06-28T20:07:28.229 INFO:teuthology.orchestra.run.smithi043.stdout:55834574852 2023-06-28T20:07:28.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.222+0000 7f17aeffd700 1 -- 172.21.15.43:0/3084930751 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f17b40697c0 con 0x7f17b808a660 2023-06-28T20:07:28.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.226+0000 7f17c078a700 1 -- 172.21.15.43:0/3084930751 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f179c06ace0 msgr2=0x7f179c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:28.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.226+0000 7f17c078a700 1 --2- 172.21.15.43:0/3084930751 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f179c06ace0 0x7f179c06d190 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f17b8164850 tx=0x7f17a8007a40 comp rx=0 tx=0).stop 2023-06-28T20:07:28.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.226+0000 7f17c078a700 1 -- 172.21.15.43:0/3084930751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17b808a660 msgr2=0x7f17b816a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:28.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.226+0000 7f17c078a700 1 --2- 172.21.15.43:0/3084930751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17b808a660 0x7f17b816a7f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f17b40209d0 tx=0x7f17b4020d90 comp rx=0 tx=0).stop 2023-06-28T20:07:28.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.226+0000 7f17c078a700 1 -- 172.21.15.43:0/3084930751 shutdown_connections 2023-06-28T20:07:28.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.226+0000 7f17c078a700 1 --2- 172.21.15.43:0/3084930751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17b80cf0b0 0x7f17b80b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.226+0000 7f17c078a700 1 --2- 172.21.15.43:0/3084930751 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17b808a660 0x7f17b816a7f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.226+0000 7f17c078a700 1 --2- 172.21.15.43:0/3084930751 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f179c06ace0 0x7f179c06d190 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.226+0000 7f17c078a700 1 --2- 172.21.15.43:0/3084930751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f17b80cde80 0x7f17b80b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.226+0000 7f17c078a700 1 -- 172.21.15.43:0/3084930751 >> 172.21.15.43:0/3084930751 conn(0x7f17b8069ef0 msgr2=0x7f17b8071290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:28.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.227+0000 7f17c078a700 1 -- 172.21.15.43:0/3084930751 shutdown_connections 2023-06-28T20:07:28.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.227+0000 7f17c078a700 1 -- 172.21.15.43:0/3084930751 wait complete. 2023-06-28T20:07:28.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.227+0000 7f5455533700 1 -- 172.21.15.43:0/3222106812 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f543c06adb0 msgr2=0x7f543c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:28.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.227+0000 7f5455533700 1 --2- 172.21.15.43:0/3222106812 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f543c06adb0 0x7f543c06d260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f545015bb20 tx=0x7f5444008040 comp rx=0 tx=0).stop 2023-06-28T20:07:28.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.227+0000 7f5455533700 1 -- 172.21.15.43:0/3222106812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5450159710 msgr2=0x7f54500ce5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:28.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.227+0000 7f5455533700 1 --2- 172.21.15.43:0/3222106812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5450159710 0x7f54500ce5f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f544001a000 tx=0x7f5440004750 comp rx=0 tx=0).stop 2023-06-28T20:07:28.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.227+0000 7f5455533700 1 -- 172.21.15.43:0/3222106812 shutdown_connections 2023-06-28T20:07:28.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.227+0000 7f5455533700 1 --2- 172.21.15.43:0/3222106812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5450159710 0x7f54500ce5f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.227+0000 7f5455533700 1 --2- 172.21.15.43:0/3222106812 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f543c06adb0 0x7f543c06d260 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.227+0000 7f5455533700 1 --2- 172.21.15.43:0/3222106812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5450158ef0 0x7f54501f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.227+0000 7f5455533700 1 --2- 172.21.15.43:0/3222106812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54501592c0 0x7f5450143460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.227+0000 7f5455533700 1 -- 172.21.15.43:0/3222106812 >> 172.21.15.43:0/3222106812 conn(0x7f5450069ef0 msgr2=0x7f545006fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:28.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.228+0000 7f5455533700 1 -- 172.21.15.43:0/3222106812 shutdown_connections 2023-06-28T20:07:28.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.228+0000 7f5455533700 1 -- 172.21.15.43:0/3222106812 wait complete. 2023-06-28T20:07:28.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.228+0000 7f66c9145700 1 -- 172.21.15.43:0/1688110000 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f66c40728b0 con 0x7f66c4202e60 2023-06-28T20:07:28.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.228+0000 7f66ab7fe700 1 -- 172.21.15.43:0/1688110000 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f66b4019090 con 0x7f66c4202e60 2023-06-28T20:07:28.235 INFO:teuthology.orchestra.run.smithi043.stdout:55834574852 2023-06-28T20:07:28.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.234+0000 7ff57a0ec700 1 -- 172.21.15.43:0/2693296331 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff5740afb50 con 0x7ff5740c9400 2023-06-28T20:07:28.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.239+0000 7ff55bfff700 1 -- 172.21.15.43:0/2693296331 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff568086e60 con 0x7ff5740c9400 2023-06-28T20:07:28.242 INFO:teuthology.orchestra.run.smithi043.stdout:55834574851 2023-06-28T20:07:28.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.239+0000 7f07e57f6700 1 -- 172.21.15.43:0/94038013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f07c0005870 con 0x7f07e0159730 2023-06-28T20:07:28.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.239+0000 7f07c77fe700 1 -- 172.21.15.43:0/94038013 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f07d8058ac0 con 0x7f07e0159730 2023-06-28T20:07:28.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.240+0000 7f66c9145700 1 -- 172.21.15.43:0/1688110000 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f66b006ad30 msgr2=0x7f66b006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:28.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.240+0000 7f66c9145700 1 --2- 172.21.15.43:0/1688110000 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f66b006ad30 0x7f66b006d1e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f66bc000f80 tx=0x7f66bc005040 comp rx=0 tx=0).stop 2023-06-28T20:07:28.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.240+0000 7f66c9145700 1 -- 172.21.15.43:0/1688110000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66c4202e60 msgr2=0x7f66c40b0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:28.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.240+0000 7f66c9145700 1 --2- 172.21.15.43:0/1688110000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66c4202e60 0x7f66c40b0dd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f66b4001670 tx=0x7f66b4001980 comp rx=0 tx=0).stop 2023-06-28T20:07:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.241+0000 7ff57a0ec700 1 -- 172.21.15.43:0/2693296331 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff56006adb0 msgr2=0x7ff56006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.241+0000 7ff57a0ec700 1 --2- 172.21.15.43:0/2693296331 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff56006adb0 0x7ff56006d260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff564005910 tx=0x7ff564009040 comp rx=0 tx=0).stop 2023-06-28T20:07:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.241+0000 7ff57a0ec700 1 -- 172.21.15.43:0/2693296331 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5740c9400 msgr2=0x7ff5740ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.241+0000 7ff57a0ec700 1 --2- 172.21.15.43:0/2693296331 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5740c9400 0x7ff5740ca870 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff568000f60 tx=0x7ff56800c670 comp rx=0 tx=0).stop 2023-06-28T20:07:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.241+0000 7ff57a0ec700 1 -- 172.21.15.43:0/2693296331 shutdown_connections 2023-06-28T20:07:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.241+0000 7ff57a0ec700 1 --2- 172.21.15.43:0/2693296331 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff56006adb0 0x7ff56006d260 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff564005910 tx=0x7ff564009040 comp rx=0 tx=0).stop 2023-06-28T20:07:28.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.241+0000 7ff57a0ec700 1 --2- 172.21.15.43:0/2693296331 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff5740c9400 0x7ff5740ca870 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.241+0000 7ff57a0ec700 1 --2- 172.21.15.43:0/2693296331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff574066e90 0x7ff5740b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.241+0000 7ff57a0ec700 1 --2- 172.21.15.43:0/2693296331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5740b0130 0x7ff5740b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.241+0000 7ff57a0ec700 1 -- 172.21.15.43:0/2693296331 >> 172.21.15.43:0/2693296331 conn(0x7ff5740cfa80 msgr2=0x7ff57406cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:28.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.241+0000 7f6746a68700 1 -- 172.21.15.43:0/3263364150 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f6740202ef0 con 0x7f67400cd5e0 2023-06-28T20:07:28.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.242+0000 7f673d7fa700 1 -- 172.21.15.43:0/3263364150 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6740202ef0 con 0x7f67400cd5e0 2023-06-28T20:07:28.247 INFO:teuthology.orchestra.run.smithi043.stdout:55834574852 2023-06-28T20:07:28.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.242+0000 7ff57a0ec700 1 -- 172.21.15.43:0/2693296331 shutdown_connections 2023-06-28T20:07:28.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.244+0000 7ff57a0ec700 1 -- 172.21.15.43:0/2693296331 wait complete. 2023-06-28T20:07:28.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.244+0000 7f6746a68700 1 -- 172.21.15.43:0/3263364150 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f672c06ace0 msgr2=0x7f672c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:28.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.244+0000 7f6746a68700 1 --2- 172.21.15.43:0/3263364150 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f672c06ace0 0x7f672c06d190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f673000e350 tx=0x7f6730014000 comp rx=0 tx=0).stop 2023-06-28T20:07:28.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.244+0000 7f6746a68700 1 -- 172.21.15.43:0/3263364150 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67400cd5e0 msgr2=0x7f67400cda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:28.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.244+0000 7f6746a68700 1 --2- 172.21.15.43:0/3263364150 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67400cd5e0 0x7f67400cda50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f673800e4c0 tx=0x7f673800e880 comp rx=0 tx=0).stop 2023-06-28T20:07:28.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.244+0000 7f6746a68700 1 -- 172.21.15.43:0/3263364150 shutdown_connections 2023-06-28T20:07:28.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.244+0000 7f6746a68700 1 --2- 172.21.15.43:0/3263364150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67400b3620 0x7f67400b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.244+0000 7f6746a68700 1 --2- 172.21.15.43:0/3263364150 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f674006d8b0 0x7f67401fe4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.244+0000 7f6746a68700 1 --2- 172.21.15.43:0/3263364150 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f672c06ace0 0x7f672c06d190 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.244+0000 7f6746a68700 1 --2- 172.21.15.43:0/3263364150 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f67400cd5e0 0x7f67400cda50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.244+0000 7f6746a68700 1 -- 172.21.15.43:0/3263364150 >> 172.21.15.43:0/3263364150 conn(0x7f6740069ef0 msgr2=0x7f67400afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:28.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.244+0000 7f6746a68700 1 -- 172.21.15.43:0/3263364150 shutdown_connections 2023-06-28T20:07:28.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.244+0000 7f6746a68700 1 -- 172.21.15.43:0/3263364150 wait complete. 2023-06-28T20:07:28.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.246+0000 7f66c9145700 1 -- 172.21.15.43:0/1688110000 shutdown_connections 2023-06-28T20:07:28.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.246+0000 7f66c9145700 1 --2- 172.21.15.43:0/1688110000 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f66b006ad30 0x7f66b006d1e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.246+0000 7f66c9145700 1 --2- 172.21.15.43:0/1688110000 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f66c40b1310 0x7f66c41fe740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.246+0000 7f66c9145700 1 --2- 172.21.15.43:0/1688110000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66c4202a90 0x7f66c41441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.246+0000 7f66c9145700 1 --2- 172.21.15.43:0/1688110000 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f66c4202e60 0x7f66c40b0dd0 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f66b4001670 tx=0x7f66b4001980 comp rx=0 tx=0).stop 2023-06-28T20:07:28.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.246+0000 7f66c9145700 1 -- 172.21.15.43:0/1688110000 >> 172.21.15.43:0/1688110000 conn(0x7f66c4069ef0 msgr2=0x7f66c4141b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:28.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.249+0000 7f66c9145700 1 -- 172.21.15.43:0/1688110000 shutdown_connections 2023-06-28T20:07:28.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.249+0000 7f66c9145700 1 -- 172.21.15.43:0/1688110000 wait complete. 2023-06-28T20:07:28.255 INFO:teuthology.orchestra.run.smithi043.stdout:55834574852 2023-06-28T20:07:28.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.251+0000 7fb9d2004700 1 -- 172.21.15.43:0/180874331 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb9cc0728b0 con 0x7fb9cc06a9d0 2023-06-28T20:07:28.256 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.0 2023-06-28T20:07:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.251+0000 7f07e57f6700 1 -- 172.21.15.43:0/94038013 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f07c806b160 msgr2=0x7f07c806d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.251+0000 7f07e57f6700 1 --2- 172.21.15.43:0/94038013 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f07c806b160 0x7f07c806d610 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f07d000d370 tx=0x7f07d0011000 comp rx=0 tx=0).stop 2023-06-28T20:07:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.251+0000 7f07e57f6700 1 -- 172.21.15.43:0/94038013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07e0159730 msgr2=0x7f07e00ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.251+0000 7f07e57f6700 1 --2- 172.21.15.43:0/94038013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07e0159730 0x7f07e00ca480 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f07d800eb70 tx=0x7f07d800ee80 comp rx=0 tx=0).stop 2023-06-28T20:07:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.251+0000 7f07e57f6700 1 -- 172.21.15.43:0/94038013 shutdown_connections 2023-06-28T20:07:28.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.251+0000 7f07e57f6700 1 --2- 172.21.15.43:0/94038013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07e0159730 0x7f07e00ca480 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.251+0000 7f07e57f6700 1 --2- 172.21.15.43:0/94038013 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f07c806b160 0x7f07c806d610 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.251+0000 7f07e57f6700 1 --2- 172.21.15.43:0/94038013 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f07e0158ef0 0x7f07e01554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.251+0000 7f07e57f6700 1 --2- 172.21.15.43:0/94038013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f07e01592c0 0x7f07e0202860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.251+0000 7f07e57f6700 1 -- 172.21.15.43:0/94038013 >> 172.21.15.43:0/94038013 conn(0x7f07e0069ef0 msgr2=0x7f07e0152560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:28.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.252+0000 7f07e57f6700 1 -- 172.21.15.43:0/94038013 shutdown_connections 2023-06-28T20:07:28.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.252+0000 7f07e57f6700 1 -- 172.21.15.43:0/94038013 wait complete. 2023-06-28T20:07:28.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.253+0000 7fb9b3fff700 1 -- 172.21.15.43:0/180874331 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb9c0020070 con 0x7fb9cc06a9d0 2023-06-28T20:07:28.260 INFO:teuthology.orchestra.run.smithi043.stdout:55834574852 2023-06-28T20:07:28.261 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2023-06-28T20:07:28.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.255+0000 7fb9d2004700 1 -- 172.21.15.43:0/180874331 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb9b806adb0 msgr2=0x7fb9b806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:28.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.255+0000 7fb9d2004700 1 --2- 172.21.15.43:0/180874331 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb9b806adb0 0x7fb9b806d260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc06d180 tx=0x7fb9c400d040 comp rx=0 tx=0).stop 2023-06-28T20:07:28.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.255+0000 7fb9d2004700 1 -- 172.21.15.43:0/180874331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9cc06a9d0 msgr2=0x7fb9cc1576e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:28.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.255+0000 7fb9d2004700 1 --2- 172.21.15.43:0/180874331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9cc06a9d0 0x7fb9cc1576e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000e9d0 tx=0x7fb9c000ece0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.256+0000 7fb9d2004700 1 -- 172.21.15.43:0/180874331 shutdown_connections 2023-06-28T20:07:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.256+0000 7fb9d2004700 1 --2- 172.21.15.43:0/180874331 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb9b806adb0 0x7fb9b806d260 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.256+0000 7fb9d2004700 1 --2- 172.21.15.43:0/180874331 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb9cc158ef0 0x7fb9cc146710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.256+0000 7fb9d2004700 1 --2- 172.21.15.43:0/180874331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9cc06a9d0 0x7fb9cc1576e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.256+0000 7fb9d2004700 1 --2- 172.21.15.43:0/180874331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9cc0ca480 0x7fb9cc144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.256+0000 7fb9d2004700 1 -- 172.21.15.43:0/180874331 >> 172.21.15.43:0/180874331 conn(0x7fb9cc069ef0 msgr2=0x7fb9cc159c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.259+0000 7fb9d2004700 1 -- 172.21.15.43:0/180874331 shutdown_connections 2023-06-28T20:07:28.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.259+0000 7fb9d2004700 1 -- 172.21.15.43:0/180874331 wait complete. 2023-06-28T20:07:28.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.262+0000 7fba7cc1d700 1 -- 172.21.15.43:0/2418895424 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fba580055d0 con 0x7fba7806cd20 2023-06-28T20:07:28.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.262+0000 7fba66ffd700 1 -- 172.21.15.43:0/2418895424 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fba68016090 con 0x7fba7806cd20 2023-06-28T20:07:28.266 INFO:teuthology.orchestra.run.smithi043.stdout:55834574853 2023-06-28T20:07:28.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.265+0000 7fba7cc1d700 1 -- 172.21.15.43:0/2418895424 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fba6006aec0 msgr2=0x7fba6006d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:28.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.265+0000 7fba7cc1d700 1 --2- 172.21.15.43:0/2418895424 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fba6006aec0 0x7fba6006d370 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fba780ca550 tx=0x7fba6c009040 comp rx=0 tx=0).stop 2023-06-28T20:07:28.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.265+0000 7fba7cc1d700 1 -- 172.21.15.43:0/2418895424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba7806cd20 msgr2=0x7fba781421e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:28.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.265+0000 7fba7cc1d700 1 --2- 172.21.15.43:0/2418895424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba7806cd20 0x7fba781421e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fba7806da90 tx=0x7fba68004750 comp rx=0 tx=0).stop 2023-06-28T20:07:28.269 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2023-06-28T20:07:28.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.266+0000 7fba7cc1d700 1 -- 172.21.15.43:0/2418895424 shutdown_connections 2023-06-28T20:07:28.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.266+0000 7fba7cc1d700 1 --2- 172.21.15.43:0/2418895424 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fba6006aec0 0x7fba6006d370 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.266+0000 7fba7cc1d700 1 --2- 172.21.15.43:0/2418895424 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba780cbac0 0x7fba780b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.266+0000 7fba7cc1d700 1 --2- 172.21.15.43:0/2418895424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba7806cd20 0x7fba781421e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.266+0000 7fba7cc1d700 1 --2- 172.21.15.43:0/2418895424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba780ca850 0x7fba780b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:28.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.266+0000 7fba7cc1d700 1 -- 172.21.15.43:0/2418895424 >> 172.21.15.43:0/2418895424 conn(0x7fba78069ef0 msgr2=0x7fba780b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:28.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.266+0000 7fba7cc1d700 1 -- 172.21.15.43:0/2418895424 shutdown_connections 2023-06-28T20:07:28.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:28.266+0000 7fba7cc1d700 1 -- 172.21.15.43:0/2418895424 wait complete. 2023-06-28T20:07:28.271 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.1 2023-06-28T20:07:28.275 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.4 2023-06-28T20:07:28.276 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.2 2023-06-28T20:07:28.285 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.3 2023-06-28T20:07:28.286 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2023-06-28T20:07:29.258 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-06-28T20:07:29.263 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-06-28T20:07:29.271 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-06-28T20:07:29.272 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-06-28T20:07:29.276 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-06-28T20:07:29.277 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-06-28T20:07:29.286 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-06-28T20:07:29.287 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-06-28T20:07:29.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.480+0000 7f620b183700 1 Processor -- start 2023-06-28T20:07:29.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.480+0000 7f620b183700 1 -- start start 2023-06-28T20:07:29.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.482+0000 7f620b183700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6204158ef0 0x7f620406cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.482+0000 7f620b183700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6204159710 0x7f62040cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.482+0000 7f620b183700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f620406d610 con 0x7f62041592c0 2023-06-28T20:07:29.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.482+0000 7f620b183700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f620415c110 con 0x7f6204158ef0 2023-06-28T20:07:29.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.482+0000 7f620b183700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620415c3f0 con 0x7f6204159710 2023-06-28T20:07:29.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.483+0000 7f6208f1f700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6204158ef0 0x7f620406cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.483+0000 7f6208f1f700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6204158ef0 0x7f620406cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:39310/0 (socket says 172.21.15.43:39310) 2023-06-28T20:07:29.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.483+0000 7f6208f1f700 1 -- 172.21.15.43:0/103673277 learned_addr learned my addr 172.21.15.43:0/103673277 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:29.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.483+0000 7f6209720700 1 --2- 172.21.15.43:0/103673277 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6204159710 0x7f62040cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.484+0000 7f6208f1f700 1 -- 172.21.15.43:0/103673277 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6204159710 msgr2=0x7f62040cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:29.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.484+0000 7f6208f1f700 1 --2- 172.21.15.43:0/103673277 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6204159710 0x7f62040cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.487 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.484+0000 7f6208f1f700 1 -- 172.21.15.43:0/103673277 >> v1:172.21.15.43:6789/0 conn(0x7f62041592c0 legacy=0x7f62040ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:29.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.484+0000 7f6208f1f700 1 -- 172.21.15.43:0/103673277 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620415c6d0 con 0x7f6204158ef0 2023-06-28T20:07:29.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.484+0000 7f6208f1f700 1 --2- 172.21.15.43:0/103673277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6204158ef0 0x7f620406cf10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009ea0 tx=0x7f61fc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3422d822e6ad1be5 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.484+0000 7f62037fe700 1 -- 172.21.15.43:0/103673277 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc003c20 con 0x7f6204158ef0 2023-06-28T20:07:29.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.484+0000 7f62037fe700 1 -- 172.21.15.43:0/103673277 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61fc003d80 con 0x7f6204158ef0 2023-06-28T20:07:29.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.485+0000 7f620b183700 1 -- 172.21.15.43:0/103673277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6204158ef0 msgr2=0x7f620406cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.485+0000 7f620b183700 1 --2- 172.21.15.43:0/103673277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6204158ef0 0x7f620406cf10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009ea0 tx=0x7f61fc00b330 comp rx=0 tx=0).stop 2023-06-28T20:07:29.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.486+0000 7f620b183700 1 -- 172.21.15.43:0/103673277 shutdown_connections 2023-06-28T20:07:29.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.486+0000 7f620b183700 1 --2- 172.21.15.43:0/103673277 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6204159710 0x7f62040cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.486+0000 7f620b183700 1 --2- 172.21.15.43:0/103673277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6204158ef0 0x7f620406cf10 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009ea0 tx=0x7f61fc00b330 comp rx=0 tx=0).stop 2023-06-28T20:07:29.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.486+0000 7f620b183700 1 -- 172.21.15.43:0/103673277 >> 172.21.15.43:0/103673277 conn(0x7f6204069ef0 msgr2=0x7f620406a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:29.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.486+0000 7f620b183700 1 -- 172.21.15.43:0/103673277 shutdown_connections 2023-06-28T20:07:29.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.486+0000 7f620b183700 1 -- 172.21.15.43:0/103673277 wait complete. 2023-06-28T20:07:29.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.488+0000 7f620b183700 1 Processor -- start 2023-06-28T20:07:29.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.488+0000 7f620b183700 1 -- start start 2023-06-28T20:07:29.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.489+0000 7f620b183700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62041592c0 0x7f620414ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.489+0000 7f620b183700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6204159710 0x7f620414c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.489+0000 7f620b183700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f620414d4a0 0x7f6204155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.489+0000 7f620b183700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62040b2370 con 0x7f620414d4a0 2023-06-28T20:07:29.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.490+0000 7f620b183700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6204154e40 con 0x7f62041592c0 2023-06-28T20:07:29.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.490+0000 7f620b183700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62040b1a40 con 0x7f6204159710 2023-06-28T20:07:29.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.490+0000 7f6209720700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f620414d4a0 0x7f6204155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.490+0000 7f6209720700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f620414d4a0 0x7f6204155270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53254/0 (socket says 172.21.15.43:53254) 2023-06-28T20:07:29.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.490+0000 7f6209720700 1 -- 172.21.15.43:0/2871116907 learned_addr learned my addr 172.21.15.43:0/2871116907 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:29.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.491+0000 7f6209720700 1 -- 172.21.15.43:0/2871116907 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6204159710 msgr2=0x7f620414c6a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-28T20:07:29.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.491+0000 7f6209720700 1 --2- 172.21.15.43:0/2871116907 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6204159710 0x7f620414c6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.491+0000 7f6209720700 1 -- 172.21.15.43:0/2871116907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62041592c0 msgr2=0x7f620414ee70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-28T20:07:29.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.491+0000 7f6209720700 1 --2- 172.21.15.43:0/2871116907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62041592c0 0x7f620414ee70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.491+0000 7f6209720700 1 -- 172.21.15.43:0/2871116907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc009b30 con 0x7f620414d4a0 2023-06-28T20:07:29.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.491+0000 7f6209720700 1 --2- 172.21.15.43:0/2871116907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f620414d4a0 0x7f6204155270 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f61f800e6f0 tx=0x7f61f800eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.495+0000 7f62017fa700 1 -- 172.21.15.43:0/2871116907 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f8013780 con 0x7f620414d4a0 2023-06-28T20:07:29.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.495+0000 7f62017fa700 1 -- 172.21.15.43:0/2871116907 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f8010040 con 0x7f620414d4a0 2023-06-28T20:07:29.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.495+0000 7f62017fa700 1 -- 172.21.15.43:0/2871116907 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f8013b20 con 0x7f620414d4a0 2023-06-28T20:07:29.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.495+0000 7f620b183700 1 -- 172.21.15.43:0/2871116907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62040b1d20 con 0x7f620414d4a0 2023-06-28T20:07:29.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.495+0000 7f620b183700 1 -- 172.21.15.43:0/2871116907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f620414cbe0 con 0x7f620414d4a0 2023-06-28T20:07:29.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.498+0000 7f620b183700 1 -- 172.21.15.43:0/2871116907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62040728b0 con 0x7f620414d4a0 2023-06-28T20:07:29.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.510+0000 7f62017fa700 1 -- 172.21.15.43:0/2871116907 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f61f8013c80 con 0x7f620414d4a0 2023-06-28T20:07:29.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.510+0000 7f62017fa700 1 --2- 172.21.15.43:0/2871116907 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f61f006adb0 0x7f61f006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.510+0000 7f62017fa700 1 -- 172.21.15.43:0/2871116907 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f61f8088c10 con 0x7f620414d4a0 2023-06-28T20:07:29.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.511+0000 7f62017fa700 1 -- 172.21.15.43:0/2871116907 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f61f808c2a0 con 0x7f620414d4a0 2023-06-28T20:07:29.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.511+0000 7f6208f1f700 1 --2- 172.21.15.43:0/2871116907 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f61f006adb0 0x7f61f006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.512+0000 7f6208f1f700 1 --2- 172.21.15.43:0/2871116907 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f61f006adb0 0x7f61f006d260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f62040cc4a0 tx=0x7f61fc009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.529+0000 7f873cb18700 1 Processor -- start 2023-06-28T20:07:29.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.529+0000 7f873cb18700 1 -- start start 2023-06-28T20:07:29.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.530+0000 7f873cb18700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f87380b2260 0x7f873806a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.530+0000 7f873cb18700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f873806b270 0x7f8738164ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.530+0000 7f873cb18700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87380b5d30 con 0x7f873806ae20 2023-06-28T20:07:29.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.530+0000 7f873cb18700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f873809b1c0 con 0x7f873806b270 2023-06-28T20:07:29.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.530+0000 7f873cb18700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f873809aec0 con 0x7f87380b2260 2023-06-28T20:07:29.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.530+0000 7f8735d9b700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f873806ae20 0x7f87381647e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42446/0 (socket says 172.21.15.43:42446) 2023-06-28T20:07:29.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.530+0000 7f8735d9b700 1 -- 172.21.15.43:0/3682593126 learned_addr learned my addr 172.21.15.43:0/3682593126 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:29.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.530+0000 7f8736d9d700 1 --2- 172.21.15.43:0/3682593126 >> v2:172.21.15.43:3300/0 conn(0x7f873806b270 0x7f8738164ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.531+0000 7f8736d9d700 1 -- 172.21.15.43:0/3682593126 >> v2:172.21.15.178:3300/0 conn(0x7f87380b2260 msgr2=0x7f873806a410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-28T20:07:29.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.531+0000 7f8736d9d700 1 --2- 172.21.15.43:0/3682593126 >> v2:172.21.15.178:3300/0 conn(0x7f87380b2260 0x7f873806a410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.531+0000 7f8736d9d700 1 -- 172.21.15.43:0/3682593126 >> v1:172.21.15.43:6789/0 conn(0x7f873806ae20 legacy=0x7f87381647e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.531+0000 7f8736d9d700 1 -- 172.21.15.43:0/3682593126 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87380d1ae0 con 0x7f873806b270 2023-06-28T20:07:29.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.531+0000 7f8736d9d700 1 --2- 172.21.15.43:0/3682593126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f873806b270 0x7f8738164ef0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f87380b5a50 tx=0x7f873000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76a3f48831bc17f server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.532+0000 7f873559a700 1 -- 172.21.15.43:0/3682593126 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873000e070 con 0x7f873806b270 2023-06-28T20:07:29.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.532+0000 7f873559a700 1 -- 172.21.15.43:0/3682593126 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f873000bad0 con 0x7f873806b270 2023-06-28T20:07:29.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.532+0000 7f873559a700 1 -- 172.21.15.43:0/3682593126 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873000be80 con 0x7f873806b270 2023-06-28T20:07:29.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.532+0000 7ff455057700 1 Processor -- start 2023-06-28T20:07:29.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.533+0000 7f873cb18700 1 -- 172.21.15.43:0/3682593126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f873806b270 msgr2=0x7f8738164ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.533+0000 7f873cb18700 1 --2- 172.21.15.43:0/3682593126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f873806b270 0x7f8738164ef0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f87380b5a50 tx=0x7f873000b330 comp rx=0 tx=0).stop 2023-06-28T20:07:29.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.533+0000 7f873cb18700 1 -- 172.21.15.43:0/3682593126 shutdown_connections 2023-06-28T20:07:29.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.533+0000 7f873cb18700 1 --2- 172.21.15.43:0/3682593126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f873806b270 0x7f8738164ef0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.533+0000 7f873cb18700 1 --2- 172.21.15.43:0/3682593126 >> v2:172.21.15.178:3300/0 conn(0x7f87380b2260 0x7f873806a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.533+0000 7f873cb18700 1 -- 172.21.15.43:0/3682593126 >> 172.21.15.43:0/3682593126 conn(0x7f87380d05c0 msgr2=0x7f87380d2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:29.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.534+0000 7f7b608a6700 1 Processor -- start 2023-06-28T20:07:29.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.534+0000 7f7b608a6700 1 -- start start 2023-06-28T20:07:29.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.533+0000 7ff455057700 1 -- start start 2023-06-28T20:07:29.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.534+0000 7ff455057700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4500c2650 0x7ff4500cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.534+0000 7ff455057700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff450164660 0x7ff45006a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.534+0000 7ff455057700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff450069ef0 con 0x7ff45006ac10 2023-06-28T20:07:29.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.534+0000 7ff455057700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff45006a2b0 con 0x7ff450164660 2023-06-28T20:07:29.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.534+0000 7ff455057700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff450164e90 con 0x7ff4500c2650 2023-06-28T20:07:29.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.534+0000 7ff44f59e700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff45006ac10 0x7ff45006c0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42452/0 (socket says 172.21.15.43:42452) 2023-06-28T20:07:29.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.534+0000 7ff44f59e700 1 -- 172.21.15.43:0/1204856985 learned_addr learned my addr 172.21.15.43:0/1204856985 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:29.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.534+0000 7f873cb18700 1 -- 172.21.15.43:0/3682593126 shutdown_connections 2023-06-28T20:07:29.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.534+0000 7ff44e59c700 1 --2- 172.21.15.43:0/1204856985 >> v2:172.21.15.178:3300/0 conn(0x7ff450164660 0x7ff45006a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.535+0000 7faf5c630700 1 Processor -- start 2023-06-28T20:07:29.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.535+0000 7ff44e59c700 1 -- 172.21.15.43:0/1204856985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4500c2650 msgr2=0x7ff4500cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:29.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.535+0000 7ff44dd9b700 1 -- 172.21.15.43:0/1204856985 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4249800244 0 0) 0x7ff450069ef0 con 0x7ff45006ac10 2023-06-28T20:07:29.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.535+0000 7ff44e59c700 1 --2- 172.21.15.43:0/1204856985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4500c2650 0x7ff4500cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.535+0000 7ff44e59c700 1 -- 172.21.15.43:0/1204856985 >> v1:172.21.15.43:6789/0 conn(0x7ff45006ac10 legacy=0x7ff45006c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.535+0000 7ff44e59c700 1 -- 172.21.15.43:0/1204856985 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff450165170 con 0x7ff450164660 2023-06-28T20:07:29.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.535+0000 7faf5c630700 1 -- start start 2023-06-28T20:07:29.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.534+0000 7f873cb18700 1 -- 172.21.15.43:0/3682593126 wait complete. 2023-06-28T20:07:29.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.535+0000 7f7b608a6700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f7b5c15cb80 0x7f7b5c06cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.535+0000 7f7b608a6700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b5c1594c0 con 0x7f7b5c0cab90 2023-06-28T20:07:29.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.535+0000 7f7b608a6700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b5c06d610 con 0x7f7b5c06d8b0 2023-06-28T20:07:29.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.535+0000 7f7b608a6700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b5c15cfc0 con 0x7f7b5c15cb80 2023-06-28T20:07:29.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.535+0000 7ff44e59c700 1 --2- 172.21.15.43:0/1204856985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff450164660 0x7ff45006a6d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff444009840 tx=0x7ff444009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=565b18a013f05d6d server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f5215ec0700 1 Processor -- start 2023-06-28T20:07:29.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f5215ec0700 1 -- start start 2023-06-28T20:07:29.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.535+0000 7f7b5a59c700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f7b5c06d8b0 0x7f7b5c0ca480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:41580/0 (socket says 172.21.15.43:41580) 2023-06-28T20:07:29.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.536+0000 7f7b5a59c700 1 -- 172.21.15.43:0/2378689914 learned_addr learned my addr 172.21.15.43:0/2378689914 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:29.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.536+0000 7f7b5959a700 1 -- 172.21.15.43:0/2378689914 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3042853414 0 0) 0x7f7b5c1594c0 con 0x7f7b5c0cab90 2023-06-28T20:07:29.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.536+0000 7f7b5959a700 1 -- 172.21.15.43:0/2378689914 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b440035b0 con 0x7f7b5c0cab90 2023-06-28T20:07:29.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.536+0000 7f7b5ad9d700 1 --2- 172.21.15.43:0/2378689914 >> v2:172.21.15.178:3300/0 conn(0x7f7b5c15cb80 0x7f7b5c06cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.536+0000 7f7b5959a700 1 -- 172.21.15.43:0/2378689914 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 701140550 0 0) 0x7f7b440035b0 con 0x7f7b5c0cab90 2023-06-28T20:07:29.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.536+0000 7f7b5959a700 1 -- 172.21.15.43:0/2378689914 >> v2:172.21.15.178:3300/0 conn(0x7f7b5c15cb80 msgr2=0x7f7b5c06cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:29.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.536+0000 7f7b5959a700 1 --2- 172.21.15.43:0/2378689914 >> v2:172.21.15.178:3300/0 conn(0x7f7b5c15cb80 0x7f7b5c06cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.536+0000 7f7b5959a700 1 -- 172.21.15.43:0/2378689914 >> v1:172.21.15.178:6789/0 conn(0x7f7b5c06d8b0 legacy=0x7f7b5c0ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:29.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.536+0000 7f7b5959a700 1 -- 172.21.15.43:0/2378689914 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5c0cd5f0 con 0x7f7b5c0cab90 2023-06-28T20:07:29.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.536+0000 7f7b5959a700 1 -- 172.21.15.43:0/2378689914 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f7b50001670 con 0x7f7b5c0cab90 2023-06-28T20:07:29.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.536+0000 7f7b5959a700 1 -- 172.21.15.43:0/2378689914 >> v1:172.21.15.43:6789/0 conn(0x7f7b5c0cab90 legacy=0x7f7b5c15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7f7b5959a700 1 --2- 172.21.15.43:0/2378689914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b44003a00 0x7f7b44003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7f7b5959a700 1 --2- 172.21.15.43:0/2378689914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b44004850 0x7f7b44004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7f7b5959a700 1 --2- 172.21.15.43:0/2378689914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b44005200 0x7f7b4400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7f7b5959a700 1 -- 172.21.15.43:0/2378689914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c15cfc0 con 0x7f7b44005200 2023-06-28T20:07:29.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7f7b5959a700 1 -- 172.21.15.43:0/2378689914 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4400c110 con 0x7f7b44003a00 2023-06-28T20:07:29.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7f7b5959a700 1 -- 172.21.15.43:0/2378689914 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4400c2d0 con 0x7f7b44004850 2023-06-28T20:07:29.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7f7b5ad9d700 1 --2- 172.21.15.43:0/2378689914 >> v2:172.21.15.178:3300/0 conn(0x7f7b5c15cb80 0x7f7b5c06cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:29.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7f7b5ad9d700 1 --2- 172.21.15.43:0/2378689914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b44005200 0x7f7b4400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7f7b5ad9d700 1 -- 172.21.15.43:0/2378689914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b44004850 msgr2=0x7f7b44004cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-28T20:07:29.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7f7b5ad9d700 1 --2- 172.21.15.43:0/2378689914 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b44004850 0x7f7b44004cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7f7b5ad9d700 1 -- 172.21.15.43:0/2378689914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b44003a00 msgr2=0x7f7b44003df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-28T20:07:29.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7f7b5ad9d700 1 --2- 172.21.15.43:0/2378689914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b44003a00 0x7f7b44003df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7f7b5ad9d700 1 -- 172.21.15.43:0/2378689914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b440035b0 con 0x7f7b44005200 2023-06-28T20:07:29.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7f7b5ad9d700 1 -- 172.21.15.43:0/2378689914 reap_dead start 2023-06-28T20:07:29.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f7b5ad9d700 1 --2- 172.21.15.43:0/2378689914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b44005200 0x7f7b4400ba10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009c30 tx=0x7f7b44005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9afc401061b560ca server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f7b5959a700 1 -- 172.21.15.43:0/2378689914 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b54007cd0 con 0x7f7b44005200 2023-06-28T20:07:29.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f7b5959a700 1 -- 172.21.15.43:0/2378689914 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b54007e30 con 0x7f7b44005200 2023-06-28T20:07:29.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f7b5959a700 1 -- 172.21.15.43:0/2378689914 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b54036610 con 0x7f7b44005200 2023-06-28T20:07:29.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.536+0000 7f873cb18700 1 Processor -- start 2023-06-28T20:07:29.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.536+0000 7f873cb18700 1 -- start start 2023-06-28T20:07:29.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f873cb18700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f873806ae20 0x7f8738150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f873cb18700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f873806b270 0x7f8738070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f873cb18700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87380b2260 0x7f87380710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f873cb18700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873809ff80 con 0x7f873806b270 2023-06-28T20:07:29.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f873cb18700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87380a0260 con 0x7f873806ae20 2023-06-28T20:07:29.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f873cb18700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87380a0540 con 0x7f87380b2260 2023-06-28T20:07:29.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.535+0000 7ff44dd9b700 1 -- 172.21.15.43:0/1204856985 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444011830 con 0x7ff450164660 2023-06-28T20:07:29.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.535+0000 7ff44dd9b700 1 -- 172.21.15.43:0/1204856985 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff444011990 con 0x7ff450164660 2023-06-28T20:07:29.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.535+0000 7ff44dd9b700 1 -- 172.21.15.43:0/1204856985 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444011d20 con 0x7ff450164660 2023-06-28T20:07:29.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.536+0000 7ff455057700 1 -- 172.21.15.43:0/1204856985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff450164660 msgr2=0x7ff45006a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.536+0000 7ff455057700 1 --2- 172.21.15.43:0/1204856985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff450164660 0x7ff45006a6d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff444009840 tx=0x7ff444009b50 comp rx=0 tx=0).stop 2023-06-28T20:07:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7ff455057700 1 -- 172.21.15.43:0/1204856985 shutdown_connections 2023-06-28T20:07:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7ff455057700 1 --2- 172.21.15.43:0/1204856985 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4500c2650 0x7ff4500cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7ff455057700 1 --2- 172.21.15.43:0/1204856985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff450164660 0x7ff45006a6d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7ff455057700 1 -- 172.21.15.43:0/1204856985 >> 172.21.15.43:0/1204856985 conn(0x7ff4500b1a50 msgr2=0x7ff4500cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7ff455057700 1 -- 172.21.15.43:0/1204856985 shutdown_connections 2023-06-28T20:07:29.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7ff455057700 1 -- 172.21.15.43:0/1204856985 wait complete. 2023-06-28T20:07:29.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f0ef0892700 1 Processor -- start 2023-06-28T20:07:29.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7faf5c630700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7faf540ce290 0x7faf54160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7faf5c630700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf541614e0 con 0x7faf540cdec0 2023-06-28T20:07:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7faf5c630700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf541617c0 con 0x7faf54164eb0 2023-06-28T20:07:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7faf5c630700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf540ca7b0 con 0x7faf540ce290 2023-06-28T20:07:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7faf59bcb700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7faf540ce290 0x7faf54160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7faf59bcb700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7faf540ce290 0x7faf54160eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:39340/0 (socket says 172.21.15.43:39340) 2023-06-28T20:07:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7faf59bcb700 1 -- 172.21.15.43:0/360278421 learned_addr learned my addr 172.21.15.43:0/360278421 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7faf593ca700 1 -- 172.21.15.43:0/360278421 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 246755584 0 0) 0x7faf541614e0 con 0x7faf540cdec0 2023-06-28T20:07:29.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7faf59bcb700 1 -- 172.21.15.43:0/360278421 >> v1:172.21.15.178:6789/0 conn(0x7faf54164eb0 legacy=0x7faf5406c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:29.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7faf59bcb700 1 -- 172.21.15.43:0/360278421 >> v1:172.21.15.43:6789/0 conn(0x7faf540cdec0 legacy=0x7faf540ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7faf59bcb700 1 -- 172.21.15.43:0/360278421 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf54161aa0 con 0x7faf540ce290 2023-06-28T20:07:29.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.537+0000 7faf59bcb700 1 --2- 172.21.15.43:0/360278421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf540ce290 0x7faf54160eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faf440098f0 tx=0x7faf44009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cd7c4f8c02bc0cf server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f8735d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f873806b270 0x7f8738070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f8735d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f873806b270 0x7f8738070ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53286/0 (socket says 172.21.15.43:53286) 2023-06-28T20:07:29.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f8735d9b700 1 -- 172.21.15.43:0/2707023066 learned_addr learned my addr 172.21.15.43:0/2707023066 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:29.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f8736d9d700 1 --2- 172.21.15.43:0/2707023066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87380b2260 0x7f87380710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f873659c700 1 --2- 172.21.15.43:0/2707023066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f873806ae20 0x7f8738150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f8735d9b700 1 -- 172.21.15.43:0/2707023066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87380b2260 msgr2=0x7f87380710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f8735d9b700 1 --2- 172.21.15.43:0/2707023066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87380b2260 0x7f87380710e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f8735d9b700 1 -- 172.21.15.43:0/2707023066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f873806ae20 msgr2=0x7f8738150bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f8735d9b700 1 --2- 172.21.15.43:0/2707023066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f873806ae20 0x7f8738150bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f8735d9b700 1 -- 172.21.15.43:0/2707023066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8730009a70 con 0x7f873806b270 2023-06-28T20:07:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.539+0000 7f873659c700 1 --2- 172.21.15.43:0/2707023066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f873806ae20 0x7f8738150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.539+0000 7f7b608a6700 1 -- 172.21.15.43:0/2378689914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b44005200 msgr2=0x7f7b4400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.539+0000 7f7b608a6700 1 --2- 172.21.15.43:0/2378689914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b44005200 0x7f7b4400ba10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009c30 tx=0x7f7b44005880 comp rx=0 tx=0).stop 2023-06-28T20:07:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.539+0000 7f5215ec0700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f521006d310 0x7f521006cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.539+0000 7f5215ec0700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f52100cfe00 0x7f52100cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.539+0000 7f5215ec0700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52100d0680 con 0x7f521015cae0 2023-06-28T20:07:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.539+0000 7f5215ec0700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f521006bb20 con 0x7f52100cfe00 2023-06-28T20:07:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.539+0000 7f5215ec0700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f521006be00 con 0x7f521006d310 2023-06-28T20:07:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.539+0000 7f520f7fe700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f521006d310 0x7f521006cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.539+0000 7f520f7fe700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f521006d310 0x7f521006cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:39356/0 (socket says 172.21.15.43:39356) 2023-06-28T20:07:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.539+0000 7f520f7fe700 1 -- 172.21.15.43:0/3520724602 learned_addr learned my addr 172.21.15.43:0/3520724602 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.539+0000 7f7b608a6700 1 -- 172.21.15.43:0/2378689914 shutdown_connections 2023-06-28T20:07:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.539+0000 7f7b608a6700 1 --2- 172.21.15.43:0/2378689914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b44005200 0x7f7b4400ba10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.539+0000 7f7b608a6700 1 -- 172.21.15.43:0/2378689914 >> 172.21.15.43:0/2378689914 conn(0x7f7b5c069ef0 msgr2=0x7f7b5c06a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.538+0000 7f0ef0892700 1 -- start start 2023-06-28T20:07:29.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.539+0000 7f0ef0892700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0eec0cdec0 0x7f0eec160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.539+0000 7f0ef0892700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0eec0ce290 0x7f0eec164eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.539+0000 7f0ef0892700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eec1614e0 con 0x7f0eec0ca0a0 2023-06-28T20:07:29.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.539+0000 7f0ef0892700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eec1653f0 con 0x7f0eec0cdec0 2023-06-28T20:07:29.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.539+0000 7f0ef0892700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec1617c0 con 0x7f0eec0ce290 2023-06-28T20:07:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.540+0000 7f520f7fe700 1 -- 172.21.15.43:0/3520724602 >> v2:172.21.15.43:3300/0 conn(0x7f52100cfe00 msgr2=0x7f52100cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.540+0000 7f520f7fe700 1 --2- 172.21.15.43:0/3520724602 >> v2:172.21.15.43:3300/0 conn(0x7f52100cfe00 0x7f52100cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.540+0000 7f520f7fe700 1 -- 172.21.15.43:0/3520724602 >> v1:172.21.15.43:6789/0 conn(0x7f521015cae0 legacy=0x7f521006d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.540+0000 7f520f7fe700 1 -- 172.21.15.43:0/3520724602 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521006c0e0 con 0x7f521006d310 2023-06-28T20:07:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.540+0000 7f7b608a6700 1 -- 172.21.15.43:0/2378689914 shutdown_connections 2023-06-28T20:07:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.540+0000 7f7b608a6700 1 -- 172.21.15.43:0/2378689914 wait complete. 2023-06-28T20:07:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.540+0000 7ff455057700 1 Processor -- start 2023-06-28T20:07:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.540+0000 7f520f7fe700 1 --2- 172.21.15.43:0/3520724602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f521006d310 0x7f521006cbc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5210066be0 tx=0x7f5200009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=930b36083de65a55 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.540+0000 7faf593ca700 1 -- 172.21.15.43:0/360278421 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44004230 con 0x7faf540ce290 2023-06-28T20:07:29.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.540+0000 7faf593ca700 1 -- 172.21.15.43:0/360278421 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf44004390 con 0x7faf540ce290 2023-06-28T20:07:29.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.540+0000 7faf593ca700 1 -- 172.21.15.43:0/360278421 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf440036a0 con 0x7faf540ce290 2023-06-28T20:07:29.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.541+0000 7f8735d9b700 1 --2- 172.21.15.43:0/2707023066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f873806b270 0x7f8738070ba0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f872800c780 tx=0x7f872800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.540+0000 7ff455057700 1 -- start start 2023-06-28T20:07:29.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.541+0000 7ff455057700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff45006ac10 0x7ff4500c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.541+0000 7ff455057700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff450164660 0x7ff4500b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.541+0000 7ff455057700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4500b3530 0x7ff4500b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.541+0000 7ff455057700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4500b2450 con 0x7ff450164660 2023-06-28T20:07:29.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.541+0000 7ff455057700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4500c7400 con 0x7ff45006ac10 2023-06-28T20:07:29.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.541+0000 7ff455057700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4500c76e0 con 0x7ff4500b3530 2023-06-28T20:07:29.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.541+0000 7ff44e59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff450164660 0x7ff4500b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.541+0000 7f0eea59c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0eec0cdec0 0x7f0eec160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.541+0000 7f0eea59c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0eec0cdec0 0x7f0eec160eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53298/0 (socket says 172.21.15.43:53298) 2023-06-28T20:07:29.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.541+0000 7f0eea59c700 1 -- 172.21.15.43:0/2030816135 learned_addr learned my addr 172.21.15.43:0/2030816135 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:29.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.541+0000 7ff44e59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff450164660 0x7ff4500b28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53306/0 (socket says 172.21.15.43:53306) 2023-06-28T20:07:29.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.541+0000 7ff44e59c700 1 -- 172.21.15.43:0/964716921 learned_addr learned my addr 172.21.15.43:0/964716921 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:29.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.541+0000 7f8726ffd700 1 -- 172.21.15.43:0/2707023066 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8728004c80 con 0x7f873806b270 2023-06-28T20:07:29.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.541+0000 7f873cb18700 1 -- 172.21.15.43:0/2707023066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87380a0820 con 0x7f873806b270 2023-06-28T20:07:29.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.541+0000 7faf5c630700 1 -- 172.21.15.43:0/360278421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf540ce290 msgr2=0x7faf54160eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.541+0000 7faf5c630700 1 --2- 172.21.15.43:0/360278421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf540ce290 0x7faf54160eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faf440098f0 tx=0x7faf44009c00 comp rx=0 tx=0).stop 2023-06-28T20:07:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.541+0000 7f873cb18700 1 -- 172.21.15.43:0/2707023066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8738071800 con 0x7f873806b270 2023-06-28T20:07:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.541+0000 7f8726ffd700 1 -- 172.21.15.43:0/2707023066 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8728004e00 con 0x7f873806b270 2023-06-28T20:07:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.541+0000 7f8726ffd700 1 -- 172.21.15.43:0/2707023066 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8728007310 con 0x7f873806b270 2023-06-28T20:07:29.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.542+0000 7f7b608a6700 1 Processor -- start 2023-06-28T20:07:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.542+0000 7f7b608a6700 1 -- start start 2023-06-28T20:07:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.542+0000 7f0eea59c700 1 -- 172.21.15.43:0/2030816135 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0eec0ce290 msgr2=0x7f0eec164eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-28T20:07:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.542+0000 7f0eea59c700 1 --2- 172.21.15.43:0/2030816135 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0eec0ce290 0x7f0eec164eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.542+0000 7f0eea59c700 1 -- 172.21.15.43:0/2030816135 >> v1:172.21.15.178:6789/0 conn(0x7f0eec0ca0a0 legacy=0x7f0eec06c6e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-28T20:07:29.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.542+0000 7f0eea59c700 1 -- 172.21.15.43:0/2030816135 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eec161aa0 con 0x7f0eec0cdec0 2023-06-28T20:07:29.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.541+0000 7ff44ed9d700 1 --2- 172.21.15.43:0/964716921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff45006ac10 0x7ff4500c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.542+0000 7ff44e59c700 1 -- 172.21.15.43:0/964716921 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4500b3530 msgr2=0x7ff4500b2e20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-28T20:07:29.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.542+0000 7ff44e59c700 1 --2- 172.21.15.43:0/964716921 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4500b3530 0x7ff4500b2e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.542+0000 7ff44e59c700 1 -- 172.21.15.43:0/964716921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff45006ac10 msgr2=0x7ff4500c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.542+0000 7ff44e59c700 1 --2- 172.21.15.43:0/964716921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff45006ac10 0x7ff4500c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.542+0000 7ff44e59c700 1 -- 172.21.15.43:0/964716921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4440094a0 con 0x7ff450164660 2023-06-28T20:07:29.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.542+0000 7ff44e59c700 1 --2- 172.21.15.43:0/964716921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff450164660 0x7ff4500b28e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff4440090b0 tx=0x7ff444004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.542+0000 7f520effd700 1 -- 172.21.15.43:0/3520724602 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5200005e30 con 0x7f521006d310 2023-06-28T20:07:29.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.542+0000 7f520effd700 1 -- 172.21.15.43:0/3520724602 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5200006030 con 0x7f521006d310 2023-06-28T20:07:29.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.542+0000 7f520effd700 1 -- 172.21.15.43:0/3520724602 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52000063e0 con 0x7f521006d310 2023-06-28T20:07:29.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.543+0000 7faf5c630700 1 -- 172.21.15.43:0/360278421 shutdown_connections 2023-06-28T20:07:29.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.543+0000 7faf5c630700 1 --2- 172.21.15.43:0/360278421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf540ce290 0x7faf54160eb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.543+0000 7faf5c630700 1 -- 172.21.15.43:0/360278421 >> 172.21.15.43:0/360278421 conn(0x7faf54069ef0 msgr2=0x7faf5406a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:29.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.543+0000 7faf5c630700 1 -- 172.21.15.43:0/360278421 shutdown_connections 2023-06-28T20:07:29.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.543+0000 7faf5c630700 1 -- 172.21.15.43:0/360278421 wait complete. 2023-06-28T20:07:29.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.543+0000 7f7b608a6700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b44005200 0x7f7b5c140600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.543+0000 7f7b608a6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b5c0b0860 0x7f7b5c087040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.543+0000 7f7b608a6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b5c0b0cd0 0x7f7b5c087580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.543+0000 7f7b608a6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c0b2800 con 0x7f7b5c0b0860 2023-06-28T20:07:29.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.543+0000 7f7b608a6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c088380 con 0x7f7b5c0b0cd0 2023-06-28T20:07:29.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.543+0000 7f7b608a6700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c13bd50 con 0x7f7b44005200 2023-06-28T20:07:29.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.543+0000 7f7b59d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b5c0b0860 0x7f7b5c087040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.543+0000 7f7b59d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b5c0b0860 0x7f7b5c087040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53312/0 (socket says 172.21.15.43:53312) 2023-06-28T20:07:29.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.543+0000 7f7b59d9b700 1 -- 172.21.15.43:0/1150978329 learned_addr learned my addr 172.21.15.43:0/1150978329 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:29.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.543+0000 7f7b5ad9d700 1 --2- 172.21.15.43:0/1150978329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b5c0b0cd0 0x7f7b5c087580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.543+0000 7ff4377fe700 1 -- 172.21.15.43:0/964716921 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4440036a0 con 0x7ff450164660 2023-06-28T20:07:29.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.543+0000 7ff4377fe700 1 -- 172.21.15.43:0/964716921 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44401f070 con 0x7ff450164660 2023-06-28T20:07:29.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.543+0000 7ff4377fe700 1 -- 172.21.15.43:0/964716921 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff444003a60 con 0x7ff450164660 2023-06-28T20:07:29.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.543+0000 7f0eea59c700 1 --2- 172.21.15.43:0/2030816135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0eec0cdec0 0x7f0eec160eb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0edc006fd0 tx=0x7f0edc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64278a292f2299d4 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7f7b59d9b700 1 -- 172.21.15.43:0/1150978329 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b44005200 msgr2=0x7f7b5c140600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7f7b59d9b700 1 --2- 172.21.15.43:0/1150978329 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b44005200 0x7f7b5c140600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7f5215ec0700 1 -- 172.21.15.43:0/3520724602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f521006d310 msgr2=0x7f521006cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7f5215ec0700 1 --2- 172.21.15.43:0/3520724602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f521006d310 0x7f521006cbc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5210066be0 tx=0x7f5200009bd0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7f7b59d9b700 1 -- 172.21.15.43:0/1150978329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b5c0b0cd0 msgr2=0x7f7b5c087580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7f7b59d9b700 1 --2- 172.21.15.43:0/1150978329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b5c0b0cd0 0x7f7b5c087580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7f7b59d9b700 1 -- 172.21.15.43:0/1150978329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5c0b1a40 con 0x7f7b5c0b0860 2023-06-28T20:07:29.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7f7b5ad9d700 1 --2- 172.21.15.43:0/1150978329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b5c0b0cd0 0x7f7b5c087580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:29.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7f7b59d9b700 1 --2- 172.21.15.43:0/1150978329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b5c0b0860 0x7f7b5c087040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000c680 tx=0x7f7b5000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7ff455057700 1 -- 172.21.15.43:0/964716921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4500c79c0 con 0x7ff450164660 2023-06-28T20:07:29.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7ff455057700 1 -- 172.21.15.43:0/964716921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4500c6530 con 0x7ff450164660 2023-06-28T20:07:29.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7f7b4affd700 1 -- 172.21.15.43:0/1150978329 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b500054e0 con 0x7f7b5c0b0860 2023-06-28T20:07:29.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7f7b4affd700 1 -- 172.21.15.43:0/1150978329 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b50003680 con 0x7f7b5c0b0860 2023-06-28T20:07:29.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7f7b4affd700 1 -- 172.21.15.43:0/1150978329 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b50003a30 con 0x7f7b5c0b0860 2023-06-28T20:07:29.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7f7b608a6700 1 -- 172.21.15.43:0/1150978329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b5c0b1d20 con 0x7f7b5c0b0860 2023-06-28T20:07:29.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7f7b608a6700 1 -- 172.21.15.43:0/1150978329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b5c0b22e0 con 0x7f7b5c0b0860 2023-06-28T20:07:29.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.543+0000 7f873cb18700 1 -- 172.21.15.43:0/2707023066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f871c0052f0 con 0x7f873806b270 2023-06-28T20:07:29.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7f8726ffd700 1 -- 172.21.15.43:0/2707023066 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8728007470 con 0x7f873806b270 2023-06-28T20:07:29.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7f8726ffd700 1 --2- 172.21.15.43:0/2707023066 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f871806ace0 0x7f871806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7f8726ffd700 1 -- 172.21.15.43:0/2707023066 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8728088aa0 con 0x7f873806b270 2023-06-28T20:07:29.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7f873659c700 1 --2- 172.21.15.43:0/2707023066 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f871806ace0 0x7f871806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7f5215ec0700 1 -- 172.21.15.43:0/3520724602 shutdown_connections 2023-06-28T20:07:29.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7f5215ec0700 1 --2- 172.21.15.43:0/3520724602 >> v2:172.21.15.43:3300/0 conn(0x7f52100cfe00 0x7f52100cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7f5215ec0700 1 --2- 172.21.15.43:0/3520724602 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f521006d310 0x7f521006cbc0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.544+0000 7f5215ec0700 1 -- 172.21.15.43:0/3520724602 >> 172.21.15.43:0/3520724602 conn(0x7f52100664b0 msgr2=0x7f52100d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:29.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.545+0000 7f5215ec0700 1 -- 172.21.15.43:0/3520724602 shutdown_connections 2023-06-28T20:07:29.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.545+0000 7f5215ec0700 1 -- 172.21.15.43:0/3520724602 wait complete. 2023-06-28T20:07:29.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.545+0000 7ff455057700 1 -- 172.21.15.43:0/964716921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff450165430 con 0x7ff450164660 2023-06-28T20:07:29.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.545+0000 7f873659c700 1 --2- 172.21.15.43:0/2707023066 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f871806ace0 0x7f871806d190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f872c006fd0 tx=0x7f872c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.546+0000 7f0ee959a700 1 -- 172.21.15.43:0/2030816135 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc005e30 con 0x7f0eec0cdec0 2023-06-28T20:07:29.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.546+0000 7f0ee959a700 1 -- 172.21.15.43:0/2030816135 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0edc006030 con 0x7f0eec0cdec0 2023-06-28T20:07:29.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.546+0000 7f0ee959a700 1 -- 172.21.15.43:0/2030816135 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc0063e0 con 0x7f0eec0cdec0 2023-06-28T20:07:29.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.547+0000 7f0ef0892700 1 -- 172.21.15.43:0/2030816135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0eec0cdec0 msgr2=0x7f0eec160eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.547+0000 7f0ef0892700 1 --2- 172.21.15.43:0/2030816135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0eec0cdec0 0x7f0eec160eb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0edc006fd0 tx=0x7f0edc009bd0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.545+0000 7faf5c630700 1 Processor -- start 2023-06-28T20:07:29.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.545+0000 7faf5c630700 1 -- start start 2023-06-28T20:07:29.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.546+0000 7faf5c630700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf540cdec0 0x7faf541583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.546+0000 7faf5c630700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf540ce290 0x7faf54163b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.546+0000 7faf5c630700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf54164eb0 0x7faf5414efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.546+0000 7faf5c630700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf541588f0 con 0x7faf540cdec0 2023-06-28T20:07:29.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.546+0000 7faf5c630700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf541640a0 con 0x7faf540ce290 2023-06-28T20:07:29.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.546+0000 7faf5c630700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf54157f20 con 0x7faf54164eb0 2023-06-28T20:07:29.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.546+0000 7faf5a3cc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf540cdec0 0x7faf541583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.546+0000 7faf5a3cc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf540cdec0 0x7faf541583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53318/0 (socket says 172.21.15.43:53318) 2023-06-28T20:07:29.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.546+0000 7faf5a3cc700 1 -- 172.21.15.43:0/1340067398 learned_addr learned my addr 172.21.15.43:0/1340067398 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:29.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.546+0000 7faf5abcd700 1 --2- 172.21.15.43:0/1340067398 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf54164eb0 0x7faf5414efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.547+0000 7faf59bcb700 1 --2- 172.21.15.43:0/1340067398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf540ce290 0x7faf54163b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.547+0000 7faf5a3cc700 1 -- 172.21.15.43:0/1340067398 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf54164eb0 msgr2=0x7faf5414efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.547+0000 7faf5a3cc700 1 --2- 172.21.15.43:0/1340067398 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf54164eb0 0x7faf5414efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.547+0000 7faf5a3cc700 1 -- 172.21.15.43:0/1340067398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf540ce290 msgr2=0x7faf54163b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.547+0000 7faf5a3cc700 1 --2- 172.21.15.43:0/1340067398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf540ce290 0x7faf54163b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.547+0000 7faf5a3cc700 1 -- 172.21.15.43:0/1340067398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf44009550 con 0x7faf540cdec0 2023-06-28T20:07:29.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.546+0000 7f7b608a6700 1 -- 172.21.15.43:0/1150978329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b3c005320 con 0x7f7b5c0b0860 2023-06-28T20:07:29.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.547+0000 7f5215ec0700 1 Processor -- start 2023-06-28T20:07:29.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.547+0000 7f5215ec0700 1 -- start start 2023-06-28T20:07:29.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.547+0000 7faf5a3cc700 1 --2- 172.21.15.43:0/1340067398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf540cdec0 0x7faf541583b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faf5000c410 tx=0x7faf5000c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.547+0000 7f0ef0892700 1 -- 172.21.15.43:0/2030816135 shutdown_connections 2023-06-28T20:07:29.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.547+0000 7f0ef0892700 1 --2- 172.21.15.43:0/2030816135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0eec0cdec0 0x7f0eec160eb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.547+0000 7f0ef0892700 1 --2- 172.21.15.43:0/2030816135 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0eec0ce290 0x7f0eec164eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.547+0000 7f0ef0892700 1 -- 172.21.15.43:0/2030816135 >> 172.21.15.43:0/2030816135 conn(0x7f0eec069ef0 msgr2=0x7f0eec06a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:29.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.547+0000 7f0ef0892700 1 -- 172.21.15.43:0/2030816135 shutdown_connections 2023-06-28T20:07:29.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.548+0000 7f5215ec0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f521006d310 0x7f52100b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.548+0000 7f5215ec0700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52100cfe00 0x7f52100b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.548+0000 7f5215ec0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f521015cae0 0x7f52101f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.548+0000 7f5215ec0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52100b2720 con 0x7f521006d310 2023-06-28T20:07:29.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.548+0000 7f5215ec0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52100b2a00 con 0x7f521015cae0 2023-06-28T20:07:29.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.548+0000 7f5215ec0700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52100b2ce0 con 0x7f52100cfe00 2023-06-28T20:07:29.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.548+0000 7f5207fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52100cfe00 0x7f52100b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.548+0000 7f520ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f521015cae0 0x7f52101f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.548+0000 7f5207fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52100cfe00 0x7f52100b1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.43:37298/0 (socket says 172.21.15.43:37298) 2023-06-28T20:07:29.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.548+0000 7f520ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f521015cae0 0x7f52101f4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:39392/0 (socket says 172.21.15.43:39392) 2023-06-28T20:07:29.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.548+0000 7f5207fff700 1 -- 172.21.15.43:0/2506603747 learned_addr learned my addr 172.21.15.43:0/2506603747 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:29.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.549+0000 7f5207fff700 1 -- 172.21.15.43:0/2506603747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f521015cae0 msgr2=0x7f52101f4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.549+0000 7f5207fff700 1 --2- 172.21.15.43:0/2506603747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f521015cae0 0x7f52101f4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.549+0000 7f5207fff700 1 -- 172.21.15.43:0/2506603747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f521006d310 msgr2=0x7f52100b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.549+0000 7f5207fff700 1 --2- 172.21.15.43:0/2506603747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f521006d310 0x7f52100b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.549+0000 7f5207fff700 1 -- 172.21.15.43:0/2506603747 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5200009500 con 0x7f52100cfe00 2023-06-28T20:07:29.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.549+0000 7f520ffff700 1 --2- 172.21.15.43:0/2506603747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f521015cae0 0x7f52101f4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:29.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.549+0000 7f5207fff700 1 --2- 172.21.15.43:0/2506603747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52100cfe00 0x7f52100b1ed0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f51f800b470 tx=0x7f51f800b780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.549+0000 7f520cff9700 1 -- 172.21.15.43:0/2506603747 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f8011830 con 0x7f52100cfe00 2023-06-28T20:07:29.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.549+0000 7f520cff9700 1 -- 172.21.15.43:0/2506603747 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f80119b0 con 0x7f52100cfe00 2023-06-28T20:07:29.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.549+0000 7f520cff9700 1 -- 172.21.15.43:0/2506603747 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f8011d60 con 0x7f52100cfe00 2023-06-28T20:07:29.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.547+0000 7f0ef0892700 1 -- 172.21.15.43:0/2030816135 wait complete. 2023-06-28T20:07:29.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.549+0000 7f0ef0892700 1 Processor -- start 2023-06-28T20:07:29.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.550+0000 7f5215ec0700 1 -- 172.21.15.43:0/2506603747 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52100b2fc0 con 0x7f52100cfe00 2023-06-28T20:07:29.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.550+0000 7f5215ec0700 1 -- 172.21.15.43:0/2506603747 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52100b2410 con 0x7f52100cfe00 2023-06-28T20:07:29.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.551+0000 7f5215ec0700 1 -- 172.21.15.43:0/2506603747 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521006a9a0 con 0x7f52100cfe00 2023-06-28T20:07:29.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.550+0000 7f0ef0892700 1 -- start start 2023-06-28T20:07:29.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.551+0000 7f0ef0892700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0eec0ca0a0 0x7f0eec13f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.551+0000 7f0ef0892700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0eec0cdec0 0x7f0eec135200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.551+0000 7f0ef0892700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0eec0ce290 0x7f0eec0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.551+0000 7f0ef0892700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec135740 con 0x7f0eec0ce290 2023-06-28T20:07:29.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.551+0000 7f0ef0892700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec0b2b10 con 0x7f0eec0cdec0 2023-06-28T20:07:29.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.551+0000 7f0ef0892700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec0b2df0 con 0x7f0eec0ca0a0 2023-06-28T20:07:29.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.551+0000 7f0eea59c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0eec0ca0a0 0x7f0eec13f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.551+0000 7f0eea59c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0eec0ca0a0 0x7f0eec13f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.43:37302/0 (socket says 172.21.15.43:37302) 2023-06-28T20:07:29.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.551+0000 7f0eea59c700 1 -- 172.21.15.43:0/965280743 learned_addr learned my addr 172.21.15.43:0/965280743 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:29.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.551+0000 7f0eead9d700 1 --2- 172.21.15.43:0/965280743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0eec0ce290 0x7f0eec0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.552+0000 7f0eead9d700 1 -- 172.21.15.43:0/965280743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0eec0ca0a0 msgr2=0x7f0eec13f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.552+0000 7f0eead9d700 1 --2- 172.21.15.43:0/965280743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0eec0ca0a0 0x7f0eec13f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.552+0000 7f0eead9d700 1 -- 172.21.15.43:0/965280743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0eec0cdec0 msgr2=0x7f0eec135200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.552+0000 7f0eead9d700 1 --2- 172.21.15.43:0/965280743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0eec0cdec0 0x7f0eec135200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.552+0000 7f0eead9d700 1 -- 172.21.15.43:0/965280743 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0edc009500 con 0x7f0eec0ce290 2023-06-28T20:07:29.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.552+0000 7f0eead9d700 1 --2- 172.21.15.43:0/965280743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0eec0ce290 0x7f0eec0b2090 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400bb00 tx=0x7f0ee400bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.552+0000 7f520cff9700 1 -- 172.21.15.43:0/2506603747 <== mon.2 v2:172.21.15.178:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f51f801a400 con 0x7f52100cfe00 2023-06-28T20:07:29.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.553+0000 7f520cff9700 1 --2- 172.21.15.43:0/2506603747 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f51fc06ae60 0x7f51fc06d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.552+0000 7faf4affd700 1 -- 172.21.15.43:0/1340067398 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf500054e0 con 0x7faf540cdec0 2023-06-28T20:07:29.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.553+0000 7faf4affd700 1 -- 172.21.15.43:0/1340067398 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5000ce20 con 0x7faf540cdec0 2023-06-28T20:07:29.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.553+0000 7faf4affd700 1 -- 172.21.15.43:0/1340067398 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf500038c0 con 0x7faf540cdec0 2023-06-28T20:07:29.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.553+0000 7faf5c630700 1 -- 172.21.15.43:0/1340067398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf541500f0 con 0x7faf540cdec0 2023-06-28T20:07:29.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.553+0000 7f520cff9700 1 -- 172.21.15.43:0/2506603747 <== mon.2 v2:172.21.15.178:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f51f8089660 con 0x7f52100cfe00 2023-06-28T20:07:29.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.553+0000 7faf5c630700 1 -- 172.21.15.43:0/1340067398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf541575f0 con 0x7faf540cdec0 2023-06-28T20:07:29.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.553+0000 7ff4377fe700 1 -- 172.21.15.43:0/964716921 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff444003bc0 con 0x7ff450164660 2023-06-28T20:07:29.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.553+0000 7f7b4affd700 1 -- 172.21.15.43:0/1150978329 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7b50003b90 con 0x7f7b5c0b0860 2023-06-28T20:07:29.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.553+0000 7ff4377fe700 1 --2- 172.21.15.43:0/964716921 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff43c06ad30 0x7ff43c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.554+0000 7f7b4affd700 1 --2- 172.21.15.43:0/1150978329 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7b44070df0 0x7f7b440732a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.554+0000 7ff4377fe700 1 -- 172.21.15.43:0/964716921 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff444021070 con 0x7ff450164660 2023-06-28T20:07:29.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.554+0000 7f7b4affd700 1 -- 172.21.15.43:0/1150978329 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7b50088e50 con 0x7f7b5c0b0860 2023-06-28T20:07:29.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.554+0000 7faf5c630700 1 -- 172.21.15.43:0/1340067398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3c0052f0 con 0x7faf540cdec0 2023-06-28T20:07:29.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.554+0000 7f7b5a59c700 1 --2- 172.21.15.43:0/1150978329 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7b44070df0 0x7f7b440732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.555+0000 7f7b5a59c700 1 --2- 172.21.15.43:0/1150978329 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7b44070df0 0x7f7b440732a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c069690 tx=0x7f7b4c007b30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.555+0000 7ff44ed9d700 1 --2- 172.21.15.43:0/964716921 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff43c06ad30 0x7ff43c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.555+0000 7ff44ed9d700 1 --2- 172.21.15.43:0/964716921 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff43c06ad30 0x7ff43c06d1e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff4500afe40 tx=0x7ff44800d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.556+0000 7f520f7fe700 1 --2- 172.21.15.43:0/2506603747 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f51fc06ae60 0x7f51fc06d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.556+0000 7f8726ffd700 1 -- 172.21.15.43:0/2707023066 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8728053ab0 con 0x7f873806b270 2023-06-28T20:07:29.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.556+0000 7f520f7fe700 1 --2- 172.21.15.43:0/2506603747 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f51fc06ae60 0x7f51fc06d310 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5210066be0 tx=0x7f5200010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.558+0000 7ff4377fe700 1 -- 172.21.15.43:0/964716921 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff444056280 con 0x7ff450164660 2023-06-28T20:07:29.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.561+0000 7f520cff9700 1 -- 172.21.15.43:0/2506603747 <== mon.2 v2:172.21.15.178:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f51f8054780 con 0x7f52100cfe00 2023-06-28T20:07:29.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.563+0000 7f0edaffd700 1 -- 172.21.15.43:0/965280743 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee400cae0 con 0x7f0eec0ce290 2023-06-28T20:07:29.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.563+0000 7f0edaffd700 1 -- 172.21.15.43:0/965280743 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee400cc40 con 0x7f0eec0ce290 2023-06-28T20:07:29.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.563+0000 7f0edaffd700 1 -- 172.21.15.43:0/965280743 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4014440 con 0x7f0eec0ce290 2023-06-28T20:07:29.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.564+0000 7f0ef0892700 1 -- 172.21.15.43:0/965280743 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eec13ead0 con 0x7f0eec0ce290 2023-06-28T20:07:29.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.564+0000 7f0ef0892700 1 -- 172.21.15.43:0/965280743 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eec13e780 con 0x7f0eec0ce290 2023-06-28T20:07:29.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.565+0000 7f0ef0892700 1 -- 172.21.15.43:0/965280743 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eec13eff0 con 0x7f0eec0ce290 2023-06-28T20:07:29.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.564+0000 7faf4affd700 1 -- 172.21.15.43:0/1340067398 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faf50003a20 con 0x7faf540cdec0 2023-06-28T20:07:29.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.565+0000 7faf4affd700 1 --2- 172.21.15.43:0/1340067398 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7faf4006ae50 0x7faf4006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.565+0000 7faf4affd700 1 -- 172.21.15.43:0/1340067398 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faf50088b50 con 0x7faf540cdec0 2023-06-28T20:07:29.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.567+0000 7f7b4affd700 1 -- 172.21.15.43:0/1150978329 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7b50053fe0 con 0x7f7b5c0b0860 2023-06-28T20:07:29.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.568+0000 7f850731b700 1 Processor -- start 2023-06-28T20:07:29.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.568+0000 7f850731b700 1 -- start start 2023-06-28T20:07:29.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.569+0000 7f850731b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f850006cf10 0x7f85000ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.569+0000 7f850731b700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85001599d0 con 0x7f85000ca9c0 2023-06-28T20:07:29.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.569+0000 7f850731b700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f850006d360 con 0x7f8500158ef0 2023-06-28T20:07:29.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.569+0000 7f850731b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850006d610 con 0x7f850006cf10 2023-06-28T20:07:29.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.570+0000 7f85050b7700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f8500158ef0 0x7f85001592c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:41654/0 (socket says 172.21.15.43:41654) 2023-06-28T20:07:29.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.570+0000 7f85050b7700 1 -- 172.21.15.43:0/3666834951 learned_addr learned my addr 172.21.15.43:0/3666834951 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:29.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.570+0000 7f85048b6700 1 --2- 172.21.15.43:0/3666834951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f850006cf10 0x7f85000ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.567+0000 7faf59bcb700 1 --2- 172.21.15.43:0/1340067398 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7faf4006ae50 0x7faf4006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.567+0000 7faf59bcb700 1 --2- 172.21.15.43:0/1340067398 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7faf4006ae50 0x7faf4006d300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faf44000c00 tx=0x7faf44008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.572+0000 7faf4affd700 1 -- 172.21.15.43:0/1340067398 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7faf50088e60 con 0x7faf540cdec0 2023-06-28T20:07:29.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.572+0000 7f84f7fff700 1 -- 172.21.15.43:0/3666834951 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 665111200 0 0) 0x7f850006d360 con 0x7f8500158ef0 2023-06-28T20:07:29.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.572+0000 7f84f7fff700 1 -- 172.21.15.43:0/3666834951 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84ec0035b0 con 0x7f8500158ef0 2023-06-28T20:07:29.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.572+0000 7f84f7fff700 1 -- 172.21.15.43:0/3666834951 <== mon.1 v1:172.21.15.178:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2836503467 0 0) 0x7f84ec0035b0 con 0x7f8500158ef0 2023-06-28T20:07:29.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.572+0000 7f84f7fff700 1 -- 172.21.15.43:0/3666834951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f850006cf10 msgr2=0x7f85000ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:29.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.572+0000 7f84f7fff700 1 --2- 172.21.15.43:0/3666834951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f850006cf10 0x7f85000ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.572+0000 7f84f7fff700 1 -- 172.21.15.43:0/3666834951 >> v1:172.21.15.43:6789/0 conn(0x7f85000ca9c0 legacy=0x7f85000cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.572+0000 7f84f7fff700 1 -- 172.21.15.43:0/3666834951 --> v1:172.21.15.178:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850015c110 con 0x7f8500158ef0 2023-06-28T20:07:29.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.572+0000 7f84f7fff700 1 -- 172.21.15.43:0/3666834951 <== mon.1 v1:172.21.15.178:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f84fc0016c0 con 0x7f8500158ef0 2023-06-28T20:07:29.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.572+0000 7f84f7fff700 1 -- 172.21.15.43:0/3666834951 >> v1:172.21.15.178:6789/0 conn(0x7f8500158ef0 legacy=0x7f85001592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.572+0000 7f84f7fff700 1 --2- 172.21.15.43:0/3666834951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84ec003a00 0x7f84ec003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.572+0000 7f84f7fff700 1 --2- 172.21.15.43:0/3666834951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84ec004850 0x7f84ec004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.572+0000 7f84f7fff700 1 --2- 172.21.15.43:0/3666834951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84ec005200 0x7f84ec00c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.572+0000 7f84f7fff700 1 -- 172.21.15.43:0/3666834951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850006d610 con 0x7f84ec003a00 2023-06-28T20:07:29.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.572+0000 7f84f7fff700 1 -- 172.21.15.43:0/3666834951 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec009a30 con 0x7f84ec005200 2023-06-28T20:07:29.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.572+0000 7f84f7fff700 1 -- 172.21.15.43:0/3666834951 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec009bf0 con 0x7f84ec004850 2023-06-28T20:07:29.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.572+0000 7f85050b7700 1 --2- 172.21.15.43:0/3666834951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84ec003a00 0x7f84ec003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.573+0000 7f85058b8700 1 --2- 172.21.15.43:0/3666834951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84ec005200 0x7f84ec00c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.573+0000 7f85050b7700 1 -- 172.21.15.43:0/3666834951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84ec004850 msgr2=0x7f84ec004cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-28T20:07:29.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.573+0000 7f85050b7700 1 --2- 172.21.15.43:0/3666834951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f84ec004850 0x7f84ec004cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.573+0000 7f85050b7700 1 -- 172.21.15.43:0/3666834951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84ec005200 msgr2=0x7f84ec00c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:29.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.573+0000 7f85050b7700 1 --2- 172.21.15.43:0/3666834951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84ec005200 0x7f84ec00c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.573+0000 7f85050b7700 1 -- 172.21.15.43:0/3666834951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850015c110 con 0x7f84ec003a00 2023-06-28T20:07:29.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.573+0000 7f85050b7700 1 --2- 172.21.15.43:0/3666834951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84ec003a00 0x7f84ec003e10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f84fc007fd0 tx=0x7f84ec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c38c63e15dc5cb4 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.575+0000 7f0edaffd700 1 -- 172.21.15.43:0/965280743 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0ee40145a0 con 0x7f0eec0ce290 2023-06-28T20:07:29.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.576+0000 7f0edaffd700 1 --2- 172.21.15.43:0/965280743 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0ecc06ad30 0x7f0ecc06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.576+0000 7f0edaffd700 1 -- 172.21.15.43:0/965280743 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0ee4089800 con 0x7f0eec0ce290 2023-06-28T20:07:29.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.577+0000 7f84f7fff700 1 -- 172.21.15.43:0/3666834951 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc0059b0 con 0x7f84ec003a00 2023-06-28T20:07:29.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.577+0000 7f84f7fff700 1 -- 172.21.15.43:0/3666834951 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84fc005b10 con 0x7f84ec003a00 2023-06-28T20:07:29.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.577+0000 7f84f7fff700 1 -- 172.21.15.43:0/3666834951 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84fc005e80 con 0x7f84ec003a00 2023-06-28T20:07:29.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.578+0000 7f0edaffd700 1 -- 172.21.15.43:0/965280743 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0ee4054a40 con 0x7f0eec0ce290 2023-06-28T20:07:29.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.578+0000 7f0eea59c700 1 --2- 172.21.15.43:0/965280743 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0ecc06ad30 0x7f0ecc06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.578+0000 7f850731b700 1 -- 172.21.15.43:0/3666834951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84ec003a00 msgr2=0x7f84ec003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.578+0000 7f850731b700 1 --2- 172.21.15.43:0/3666834951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84ec003a00 0x7f84ec003e10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f84fc007fd0 tx=0x7f84ec005880 comp rx=0 tx=0).stop 2023-06-28T20:07:29.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.584+0000 7f85058b8700 1 -- 172.21.15.43:0/3666834951 reap_dead start 2023-06-28T20:07:29.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.584+0000 7f850731b700 1 -- 172.21.15.43:0/3666834951 shutdown_connections 2023-06-28T20:07:29.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.584+0000 7f850731b700 1 -- 172.21.15.43:0/3666834951 >> 172.21.15.43:0/3666834951 conn(0x7f8500069ef0 msgr2=0x7f850006a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:29.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.584+0000 7f850731b700 1 -- 172.21.15.43:0/3666834951 shutdown_connections 2023-06-28T20:07:29.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.585+0000 7f850731b700 1 -- 172.21.15.43:0/3666834951 wait complete. 2023-06-28T20:07:29.630 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.587+0000 7f850731b700 1 Processor -- start 2023-06-28T20:07:29.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.587+0000 7f850731b700 1 -- start start 2023-06-28T20:07:29.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.588+0000 7f850731b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f850014e900 0x7f850014f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.588+0000 7f850731b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f850014ecd0 0x7f85001fa7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.588+0000 7f850731b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8500156be0 0x7f8500157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.588+0000 7f850731b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8500157590 con 0x7f8500156be0 2023-06-28T20:07:29.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.588+0000 7f850731b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8500150600 con 0x7f850014ecd0 2023-06-28T20:07:29.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.588+0000 7f850731b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85001facf0 con 0x7f850014e900 2023-06-28T20:07:29.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.584+0000 7f0eea59c700 1 --2- 172.21.15.43:0/965280743 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0ecc06ad30 0x7f0ecc06d1e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0edc013040 tx=0x7f0edc008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.589+0000 7f85058b8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8500156be0 0x7f8500157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.589+0000 7f85058b8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8500156be0 0x7f8500157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53360/0 (socket says 172.21.15.43:53360) 2023-06-28T20:07:29.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.589+0000 7f85058b8700 1 -- 172.21.15.43:0/686223912 learned_addr learned my addr 172.21.15.43:0/686223912 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:29.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.589+0000 7f85058b8700 1 -- 172.21.15.43:0/686223912 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f850014e900 msgr2=0x7f850014f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.589+0000 7f85058b8700 1 --2- 172.21.15.43:0/686223912 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f850014e900 0x7f850014f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.589+0000 7f85058b8700 1 -- 172.21.15.43:0/686223912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f850014ecd0 msgr2=0x7f85001fa7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.589+0000 7f85058b8700 1 --2- 172.21.15.43:0/686223912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f850014ecd0 0x7f85001fa7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.589+0000 7f85058b8700 1 -- 172.21.15.43:0/686223912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850014f9f0 con 0x7f8500156be0 2023-06-28T20:07:29.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.590+0000 7f85058b8700 1 --2- 172.21.15.43:0/686223912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8500156be0 0x7f8500157050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f85000c9010 tx=0x7f84f000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.592+0000 7f84f5ffb700 1 -- 172.21.15.43:0/686223912 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f000b070 con 0x7f8500156be0 2023-06-28T20:07:29.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.592+0000 7f84f5ffb700 1 -- 172.21.15.43:0/686223912 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f00052d0 con 0x7f8500156be0 2023-06-28T20:07:29.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.592+0000 7f84f5ffb700 1 -- 172.21.15.43:0/686223912 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f0003680 con 0x7f8500156be0 2023-06-28T20:07:29.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.592+0000 7f850731b700 1 -- 172.21.15.43:0/686223912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850014fcd0 con 0x7f8500156be0 2023-06-28T20:07:29.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.592+0000 7f850731b700 1 -- 172.21.15.43:0/686223912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8500150290 con 0x7f8500156be0 2023-06-28T20:07:29.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.596+0000 7f84f5ffb700 1 -- 172.21.15.43:0/686223912 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f84f00037e0 con 0x7f8500156be0 2023-06-28T20:07:29.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.597+0000 7f84f5ffb700 1 --2- 172.21.15.43:0/686223912 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f84e406ade0 0x7f84e406d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:29.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.597+0000 7f84f5ffb700 1 -- 172.21.15.43:0/686223912 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f84f0088870 con 0x7f8500156be0 2023-06-28T20:07:29.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.598+0000 7f85050b7700 1 --2- 172.21.15.43:0/686223912 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f84e406ade0 0x7f84e406d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:29.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.598+0000 7f85050b7700 1 --2- 172.21.15.43:0/686223912 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f84e406ade0 0x7f84e406d290 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f84fc001980 tx=0x7f84fc00a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:29.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.599+0000 7f850731b700 1 -- 172.21.15.43:0/686223912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e00052f0 con 0x7f8500156be0 2023-06-28T20:07:29.637 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.608+0000 7f84f5ffb700 1 -- 172.21.15.43:0/686223912 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f84f0053870 con 0x7f8500156be0 2023-06-28T20:07:29.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.919+0000 7f620b183700 1 -- 172.21.15.43:0/2871116907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f62040ca870 con 0x7f620414d4a0 2023-06-28T20:07:29.927 INFO:teuthology.orchestra.run.smithi043.stdout:55834574853 2023-06-28T20:07:29.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.920+0000 7f62017fa700 1 -- 172.21.15.43:0/2871116907 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f61f8053ca0 con 0x7f620414d4a0 2023-06-28T20:07:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.924+0000 7f620b183700 1 -- 172.21.15.43:0/2871116907 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f61f006adb0 msgr2=0x7f61f006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.924+0000 7f620b183700 1 --2- 172.21.15.43:0/2871116907 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f61f006adb0 0x7f61f006d260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f62040cc4a0 tx=0x7f61fc009000 comp rx=0 tx=0).stop 2023-06-28T20:07:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.924+0000 7f620b183700 1 -- 172.21.15.43:0/2871116907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f620414d4a0 msgr2=0x7f6204155270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.924+0000 7f620b183700 1 --2- 172.21.15.43:0/2871116907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f620414d4a0 0x7f6204155270 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f61f800e6f0 tx=0x7f61f800eab0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.924+0000 7f620b183700 1 -- 172.21.15.43:0/2871116907 shutdown_connections 2023-06-28T20:07:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.924+0000 7f620b183700 1 --2- 172.21.15.43:0/2871116907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f620414d4a0 0x7f6204155270 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.924+0000 7f620b183700 1 --2- 172.21.15.43:0/2871116907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62041592c0 0x7f620414ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.924+0000 7f620b183700 1 --2- 172.21.15.43:0/2871116907 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f61f006adb0 0x7f61f006d260 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.924+0000 7f620b183700 1 --2- 172.21.15.43:0/2871116907 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6204159710 0x7f620414c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.924+0000 7f620b183700 1 -- 172.21.15.43:0/2871116907 >> 172.21.15.43:0/2871116907 conn(0x7f6204069ef0 msgr2=0x7f6204070f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:29.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.925+0000 7f620b183700 1 -- 172.21.15.43:0/2871116907 shutdown_connections 2023-06-28T20:07:29.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.928+0000 7f620b183700 1 -- 172.21.15.43:0/2871116907 wait complete. 2023-06-28T20:07:29.949 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2023-06-28T20:07:29.950 DEBUG:teuthology.parallel:result is None 2023-06-28T20:07:29.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.959+0000 7f7b608a6700 1 -- 172.21.15.43:0/1150978329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7b3c0058a0 con 0x7f7b5c0b0860 2023-06-28T20:07:29.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.960+0000 7f7b4affd700 1 -- 172.21.15.43:0/1150978329 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7b50017020 con 0x7f7b5c0b0860 2023-06-28T20:07:29.963 INFO:teuthology.orchestra.run.smithi043.stdout:55834574853 2023-06-28T20:07:29.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.966+0000 7f7b608a6700 1 -- 172.21.15.43:0/1150978329 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7b44070df0 msgr2=0x7f7b440732a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.966+0000 7f7b608a6700 1 --2- 172.21.15.43:0/1150978329 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7b44070df0 0x7f7b440732a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c069690 tx=0x7f7b4c007b30 comp rx=0 tx=0).stop 2023-06-28T20:07:29.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.966+0000 7f7b608a6700 1 -- 172.21.15.43:0/1150978329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b5c0b0860 msgr2=0x7f7b5c087040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.966+0000 7f7b608a6700 1 --2- 172.21.15.43:0/1150978329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b5c0b0860 0x7f7b5c087040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7b5000c680 tx=0x7f7b5000c990 comp rx=0 tx=0).stop 2023-06-28T20:07:29.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.966+0000 7f7b608a6700 1 -- 172.21.15.43:0/1150978329 shutdown_connections 2023-06-28T20:07:29.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.966+0000 7f7b608a6700 1 --2- 172.21.15.43:0/1150978329 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b5c0b0cd0 0x7f7b5c087580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.966+0000 7f7b608a6700 1 --2- 172.21.15.43:0/1150978329 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7b44070df0 0x7f7b440732a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.966+0000 7f7b608a6700 1 --2- 172.21.15.43:0/1150978329 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b44005200 0x7f7b5c140600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.966+0000 7f7b608a6700 1 --2- 172.21.15.43:0/1150978329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b5c0b0860 0x7f7b5c087040 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.966+0000 7f7b608a6700 1 -- 172.21.15.43:0/1150978329 >> 172.21.15.43:0/1150978329 conn(0x7f7b5c069ef0 msgr2=0x7f7b5c0ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:29.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.966+0000 7f7b608a6700 1 -- 172.21.15.43:0/1150978329 shutdown_connections 2023-06-28T20:07:29.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.966+0000 7f7b608a6700 1 -- 172.21.15.43:0/1150978329 wait complete. 2023-06-28T20:07:29.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.970+0000 7f873cb18700 1 -- 172.21.15.43:0/2707023066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f871c0055a0 con 0x7f873806b270 2023-06-28T20:07:29.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.971+0000 7ff455057700 1 -- 172.21.15.43:0/964716921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff450165400 con 0x7ff450164660 2023-06-28T20:07:29.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.971+0000 7f8726ffd700 1 -- 172.21.15.43:0/2707023066 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8728018090 con 0x7f873806b270 2023-06-28T20:07:29.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.971+0000 7ff4377fe700 1 -- 172.21.15.43:0/964716921 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff4440598a0 con 0x7ff450164660 2023-06-28T20:07:29.975 INFO:teuthology.orchestra.run.smithi043.stdout:55834574852 2023-06-28T20:07:29.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.975+0000 7f5215ec0700 1 -- 172.21.15.43:0/2506603747 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f52100b34e0 con 0x7f52100cfe00 2023-06-28T20:07:29.978 INFO:teuthology.orchestra.run.smithi043.stdout:55834574853 2023-06-28T20:07:29.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.975+0000 7f520cff9700 1 -- 172.21.15.43:0/2506603747 <== mon.2 v2:172.21.15.178:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f51f8057da0 con 0x7f52100cfe00 2023-06-28T20:07:29.978 INFO:teuthology.orchestra.run.smithi043.stdout:55834574855 2023-06-28T20:07:29.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.978+0000 7ff455057700 1 -- 172.21.15.43:0/964716921 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff43c06ad30 msgr2=0x7ff43c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.978+0000 7ff455057700 1 --2- 172.21.15.43:0/964716921 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff43c06ad30 0x7ff43c06d1e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff4500afe40 tx=0x7ff44800d040 comp rx=0 tx=0).stop 2023-06-28T20:07:29.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.978+0000 7ff455057700 1 -- 172.21.15.43:0/964716921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff450164660 msgr2=0x7ff4500b28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.978+0000 7ff455057700 1 --2- 172.21.15.43:0/964716921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff450164660 0x7ff4500b28e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff4440090b0 tx=0x7ff444004750 comp rx=0 tx=0).stop 2023-06-28T20:07:29.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.979+0000 7ff455057700 1 -- 172.21.15.43:0/964716921 shutdown_connections 2023-06-28T20:07:29.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.979+0000 7ff455057700 1 --2- 172.21.15.43:0/964716921 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff43c06ad30 0x7ff43c06d1e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.979+0000 7ff455057700 1 --2- 172.21.15.43:0/964716921 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4500b3530 0x7ff4500b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.979+0000 7ff455057700 1 --2- 172.21.15.43:0/964716921 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff45006ac10 0x7ff4500c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.979+0000 7ff455057700 1 --2- 172.21.15.43:0/964716921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff450164660 0x7ff4500b28e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.979+0000 7ff455057700 1 -- 172.21.15.43:0/964716921 >> 172.21.15.43:0/964716921 conn(0x7ff4500b1a50 msgr2=0x7ff450070070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:29.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.979+0000 7ff455057700 1 -- 172.21.15.43:0/964716921 shutdown_connections 2023-06-28T20:07:29.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.979+0000 7f873cb18700 1 -- 172.21.15.43:0/2707023066 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f871806ace0 msgr2=0x7f871806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.979+0000 7f873cb18700 1 --2- 172.21.15.43:0/2707023066 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f871806ace0 0x7f871806d190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f872c006fd0 tx=0x7f872c008040 comp rx=0 tx=0).stop 2023-06-28T20:07:29.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.979+0000 7f873cb18700 1 -- 172.21.15.43:0/2707023066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f873806b270 msgr2=0x7f8738070ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.979+0000 7f873cb18700 1 --2- 172.21.15.43:0/2707023066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f873806b270 0x7f8738070ba0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f872800c780 tx=0x7f872800cb40 comp rx=0 tx=0).stop 2023-06-28T20:07:29.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.979+0000 7ff455057700 1 -- 172.21.15.43:0/964716921 wait complete. 2023-06-28T20:07:29.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.979+0000 7f5215ec0700 1 -- 172.21.15.43:0/2506603747 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f51fc06ae60 msgr2=0x7f51fc06d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.979+0000 7f5215ec0700 1 --2- 172.21.15.43:0/2506603747 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f51fc06ae60 0x7f51fc06d310 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5210066be0 tx=0x7f5200010000 comp rx=0 tx=0).stop 2023-06-28T20:07:29.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.979+0000 7f5215ec0700 1 -- 172.21.15.43:0/2506603747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52100cfe00 msgr2=0x7f52100b1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:29.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.979+0000 7f5215ec0700 1 --2- 172.21.15.43:0/2506603747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52100cfe00 0x7f52100b1ed0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f51f800b470 tx=0x7f51f800b780 comp rx=0 tx=0).stop 2023-06-28T20:07:29.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.980+0000 7f873cb18700 1 -- 172.21.15.43:0/2707023066 shutdown_connections 2023-06-28T20:07:29.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.980+0000 7f873cb18700 1 --2- 172.21.15.43:0/2707023066 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f871806ace0 0x7f871806d190 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.980+0000 7f873cb18700 1 --2- 172.21.15.43:0/2707023066 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f87380b2260 0x7f87380710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.980+0000 7f873cb18700 1 --2- 172.21.15.43:0/2707023066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f873806ae20 0x7f8738150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.980+0000 7f873cb18700 1 --2- 172.21.15.43:0/2707023066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f873806b270 0x7f8738070ba0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.980+0000 7f873cb18700 1 -- 172.21.15.43:0/2707023066 >> 172.21.15.43:0/2707023066 conn(0x7f87380d05c0 msgr2=0x7f873809b650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:29.989 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2023-06-28T20:07:29.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.982+0000 7f873cb18700 1 -- 172.21.15.43:0/2707023066 shutdown_connections 2023-06-28T20:07:29.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.982+0000 7f873cb18700 1 -- 172.21.15.43:0/2707023066 wait complete. 2023-06-28T20:07:29.991 DEBUG:teuthology.parallel:result is None 2023-06-28T20:07:29.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.985+0000 7f5215ec0700 1 -- 172.21.15.43:0/2506603747 shutdown_connections 2023-06-28T20:07:29.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.985+0000 7f5215ec0700 1 --2- 172.21.15.43:0/2506603747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f521015cae0 0x7f52101f4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.985+0000 7f5215ec0700 1 --2- 172.21.15.43:0/2506603747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f521006d310 0x7f52100b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.985+0000 7f5215ec0700 1 --2- 172.21.15.43:0/2506603747 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f51fc06ae60 0x7f51fc06d310 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.985+0000 7f5215ec0700 1 --2- 172.21.15.43:0/2506603747 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52100cfe00 0x7f52100b1ed0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:29.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.985+0000 7f5215ec0700 1 -- 172.21.15.43:0/2506603747 >> 172.21.15.43:0/2506603747 conn(0x7f52100664b0 msgr2=0x7f52100ce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:29.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.987+0000 7f5215ec0700 1 -- 172.21.15.43:0/2506603747 shutdown_connections 2023-06-28T20:07:29.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.987+0000 7f5215ec0700 1 -- 172.21.15.43:0/2506603747 wait complete. 2023-06-28T20:07:29.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.992+0000 7faf5c630700 1 -- 172.21.15.43:0/1340067398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7faf3c005870 con 0x7faf540cdec0 2023-06-28T20:07:29.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.992+0000 7faf4affd700 1 -- 172.21.15.43:0/1340067398 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faf50017090 con 0x7faf540cdec0 2023-06-28T20:07:29.998 INFO:teuthology.orchestra.run.smithi043.stdout:55834574853 2023-06-28T20:07:30.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.997+0000 7f0ef0892700 1 -- 172.21.15.43:0/965280743 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0eec13eff0 con 0x7f0eec0ce290 2023-06-28T20:07:30.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.997+0000 7faf5c630700 1 -- 172.21.15.43:0/1340067398 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7faf4006ae50 msgr2=0x7faf4006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:30.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.997+0000 7faf5c630700 1 --2- 172.21.15.43:0/1340067398 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7faf4006ae50 0x7faf4006d300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faf44000c00 tx=0x7faf44008030 comp rx=0 tx=0).stop 2023-06-28T20:07:30.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.997+0000 7faf5c630700 1 -- 172.21.15.43:0/1340067398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf540cdec0 msgr2=0x7faf541583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:30.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.997+0000 7faf5c630700 1 --2- 172.21.15.43:0/1340067398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf540cdec0 0x7faf541583b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faf5000c410 tx=0x7faf5000c720 comp rx=0 tx=0).stop 2023-06-28T20:07:30.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.997+0000 7f0edaffd700 1 -- 172.21.15.43:0/965280743 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0eec13eff0 con 0x7f0eec0ce290 2023-06-28T20:07:30.001 INFO:teuthology.orchestra.run.smithi043.stdout:55834574853 2023-06-28T20:07:30.002 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.4 2023-06-28T20:07:30.003 DEBUG:teuthology.parallel:result is None 2023-06-28T20:07:30.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.999+0000 7faf5c630700 1 -- 172.21.15.43:0/1340067398 shutdown_connections 2023-06-28T20:07:30.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.999+0000 7faf5c630700 1 --2- 172.21.15.43:0/1340067398 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7faf4006ae50 0x7faf4006d300 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.999+0000 7faf5c630700 1 --2- 172.21.15.43:0/1340067398 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faf54164eb0 0x7faf5414efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.999+0000 7faf5c630700 1 --2- 172.21.15.43:0/1340067398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf540cdec0 0x7faf541583b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.999+0000 7faf5c630700 1 --2- 172.21.15.43:0/1340067398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf540ce290 0x7faf54163b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.999+0000 7faf5c630700 1 -- 172.21.15.43:0/1340067398 >> 172.21.15.43:0/1340067398 conn(0x7faf54069ef0 msgr2=0x7faf5415ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:30.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.999+0000 7faf5c630700 1 -- 172.21.15.43:0/1340067398 shutdown_connections 2023-06-28T20:07:30.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.999+0000 7faf5c630700 1 -- 172.21.15.43:0/1340067398 wait complete. 2023-06-28T20:07:30.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.999+0000 7f0ef0892700 1 -- 172.21.15.43:0/965280743 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0ecc06ad30 msgr2=0x7f0ecc06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:30.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.999+0000 7f0ef0892700 1 --2- 172.21.15.43:0/965280743 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0ecc06ad30 0x7f0ecc06d1e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0edc013040 tx=0x7f0edc008070 comp rx=0 tx=0).stop 2023-06-28T20:07:30.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.999+0000 7f0ef0892700 1 -- 172.21.15.43:0/965280743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0eec0ce290 msgr2=0x7f0eec0b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:30.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:29.999+0000 7f0ef0892700 1 --2- 172.21.15.43:0/965280743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0eec0ce290 0x7f0eec0b2090 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400bb00 tx=0x7f0ee400bec0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.000+0000 7f0ef0892700 1 -- 172.21.15.43:0/965280743 shutdown_connections 2023-06-28T20:07:30.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.000+0000 7f0ef0892700 1 --2- 172.21.15.43:0/965280743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0eec0ce290 0x7f0eec0b2090 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.000+0000 7f0ef0892700 1 --2- 172.21.15.43:0/965280743 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0ecc06ad30 0x7f0ecc06d1e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.000+0000 7f0ef0892700 1 --2- 172.21.15.43:0/965280743 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0eec0ca0a0 0x7f0eec13f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.000+0000 7f0ef0892700 1 --2- 172.21.15.43:0/965280743 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0eec0cdec0 0x7f0eec135200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.000+0000 7f0ef0892700 1 -- 172.21.15.43:0/965280743 >> 172.21.15.43:0/965280743 conn(0x7f0eec069ef0 msgr2=0x7f0eec150de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:30.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.000+0000 7f0ef0892700 1 -- 172.21.15.43:0/965280743 shutdown_connections 2023-06-28T20:07:30.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.000+0000 7f0ef0892700 1 -- 172.21.15.43:0/965280743 wait complete. 2023-06-28T20:07:30.010 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.1 2023-06-28T20:07:30.010 DEBUG:teuthology.parallel:result is None 2023-06-28T20:07:30.013 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2023-06-28T20:07:30.013 DEBUG:teuthology.parallel:result is None 2023-06-28T20:07:30.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.012+0000 7f850731b700 1 -- 172.21.15.43:0/686223912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f84e0005160 con 0x7f8500156be0 2023-06-28T20:07:30.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.012+0000 7f84f5ffb700 1 -- 172.21.15.43:0/686223912 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f84f0018030 con 0x7f8500156be0 2023-06-28T20:07:30.015 INFO:teuthology.orchestra.run.smithi043.stdout:55834574854 2023-06-28T20:07:30.018 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2023-06-28T20:07:30.019 DEBUG:teuthology.parallel:result is None 2023-06-28T20:07:30.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.015+0000 7f850731b700 1 -- 172.21.15.43:0/686223912 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f84e406ade0 msgr2=0x7f84e406d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:30.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.015+0000 7f850731b700 1 --2- 172.21.15.43:0/686223912 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f84e406ade0 0x7f84e406d290 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f84fc001980 tx=0x7f84fc00a000 comp rx=0 tx=0).stop 2023-06-28T20:07:30.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.015+0000 7f850731b700 1 -- 172.21.15.43:0/686223912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8500156be0 msgr2=0x7f8500157050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:30.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.015+0000 7f850731b700 1 --2- 172.21.15.43:0/686223912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8500156be0 0x7f8500157050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f85000c9010 tx=0x7f84f000d710 comp rx=0 tx=0).stop 2023-06-28T20:07:30.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.015+0000 7f850731b700 1 -- 172.21.15.43:0/686223912 shutdown_connections 2023-06-28T20:07:30.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.015+0000 7f850731b700 1 --2- 172.21.15.43:0/686223912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8500156be0 0x7f8500157050 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.015+0000 7f850731b700 1 --2- 172.21.15.43:0/686223912 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f84e406ade0 0x7f84e406d290 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.015+0000 7f850731b700 1 --2- 172.21.15.43:0/686223912 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f850014e900 0x7f850014f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.015+0000 7f850731b700 1 --2- 172.21.15.43:0/686223912 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f850014ecd0 0x7f85001fa7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.015+0000 7f850731b700 1 -- 172.21.15.43:0/686223912 >> 172.21.15.43:0/686223912 conn(0x7f8500069ef0 msgr2=0x7f8500157b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:30.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.015+0000 7f850731b700 1 -- 172.21.15.43:0/686223912 shutdown_connections 2023-06-28T20:07:30.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.015+0000 7f850731b700 1 -- 172.21.15.43:0/686223912 wait complete. 2023-06-28T20:07:30.021 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2023-06-28T20:07:30.022 DEBUG:teuthology.parallel:result is None 2023-06-28T20:07:30.030 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2023-06-28T20:07:30.030 DEBUG:teuthology.parallel:result is None 2023-06-28T20:07:30.031 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-06-28T20:07:30.031 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-28T20:07:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.193+0000 7f9bdde82700 1 Processor -- start 2023-06-28T20:07:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.193+0000 7f9bdde82700 1 -- start start 2023-06-28T20:07:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.194+0000 7f9bdde82700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9bd815d9f0 0x7f9bd806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:30.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.194+0000 7f9bdde82700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bd8071170 con 0x7f9bd8070cb0 2023-06-28T20:07:30.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.194+0000 7f9bdde82700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bd8071450 con 0x7f9bd806a600 2023-06-28T20:07:30.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.194+0000 7f9bdde82700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd815ddc0 con 0x7f9bd815d9f0 2023-06-28T20:07:30.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.194+0000 7f9bd77fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9bd815d9f0 0x7f9bd806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:30.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.194+0000 7f9bd6ffd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9bd8070cb0 0x7f9bd8069ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42550/0 (socket says 172.21.15.43:42550) 2023-06-28T20:07:30.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.194+0000 7f9bd6ffd700 1 -- 172.21.15.43:0/721421273 learned_addr learned my addr 172.21.15.43:0/721421273 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:30.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.195+0000 7f9bd77fe700 1 -- 172.21.15.43:0/721421273 >> v1:172.21.15.178:6789/0 conn(0x7f9bd806a600 legacy=0x7f9bd80cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:30.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.195+0000 7f9bd77fe700 1 -- 172.21.15.43:0/721421273 >> v1:172.21.15.43:6789/0 conn(0x7f9bd8070cb0 legacy=0x7f9bd8069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:30.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.195+0000 7f9bd77fe700 1 -- 172.21.15.43:0/721421273 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd80cb240 con 0x7f9bd815d9f0 2023-06-28T20:07:30.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.195+0000 7f9bd67fc700 1 -- 172.21.15.43:0/721421273 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1742848345 0 0) 0x7f9bd8071170 con 0x7f9bd8070cb0 2023-06-28T20:07:30.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.195+0000 7f9bd77fe700 1 --2- 172.21.15.43:0/721421273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bd815d9f0 0x7f9bd806d3b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009840 tx=0x7f9bc0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c2bae57df0dd77 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:30.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.195+0000 7f9bd67fc700 1 -- 172.21.15.43:0/721421273 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0011830 con 0x7f9bd815d9f0 2023-06-28T20:07:30.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.195+0000 7f9bd67fc700 1 -- 172.21.15.43:0/721421273 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc0011990 con 0x7f9bd815d9f0 2023-06-28T20:07:30.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.195+0000 7f9bd67fc700 1 -- 172.21.15.43:0/721421273 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0011d20 con 0x7f9bd815d9f0 2023-06-28T20:07:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.195+0000 7f9bdde82700 1 -- 172.21.15.43:0/721421273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bd815d9f0 msgr2=0x7f9bd806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.195+0000 7f9bdde82700 1 --2- 172.21.15.43:0/721421273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bd815d9f0 0x7f9bd806d3b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009840 tx=0x7f9bc0009b50 comp rx=0 tx=0).stop 2023-06-28T20:07:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.196+0000 7f9bdde82700 1 -- 172.21.15.43:0/721421273 shutdown_connections 2023-06-28T20:07:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.196+0000 7f9bdde82700 1 --2- 172.21.15.43:0/721421273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bd815d9f0 0x7f9bd806d3b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.196+0000 7f9bdde82700 1 -- 172.21.15.43:0/721421273 >> 172.21.15.43:0/721421273 conn(0x7f9bd80b1720 msgr2=0x7f9bd80b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.196+0000 7f9bdde82700 1 -- 172.21.15.43:0/721421273 shutdown_connections 2023-06-28T20:07:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.196+0000 7f9bdde82700 1 -- 172.21.15.43:0/721421273 wait complete. 2023-06-28T20:07:30.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.197+0000 7f9bdde82700 1 Processor -- start 2023-06-28T20:07:30.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.197+0000 7f9bdde82700 1 -- start start 2023-06-28T20:07:30.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.198+0000 7f9bdde82700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bd806a600 0x7f9bd81f30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:30.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.198+0000 7f9bdde82700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9bd8070cb0 0x7f9bd80c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:30.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.198+0000 7f9bdde82700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bd815d9f0 0x7f9bd815be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:30.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.198+0000 7f9bdde82700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd81f35e0 con 0x7f9bd815d9f0 2023-06-28T20:07:30.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.198+0000 7f9bdde82700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd80c8cf0 con 0x7f9bd806a600 2023-06-28T20:07:30.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.198+0000 7f9bdde82700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd80c8fd0 con 0x7f9bd8070cb0 2023-06-28T20:07:30.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.198+0000 7f9bd7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bd815d9f0 0x7f9bd815be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:30.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.198+0000 7f9bd7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bd815d9f0 0x7f9bd815be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53372/0 (socket says 172.21.15.43:53372) 2023-06-28T20:07:30.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.198+0000 7f9bd7fff700 1 -- 172.21.15.43:0/2387695772 learned_addr learned my addr 172.21.15.43:0/2387695772 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:30.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.198+0000 7f9bd77fe700 1 --2- 172.21.15.43:0/2387695772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bd806a600 0x7f9bd81f30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:30.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.198+0000 7f9bd6ffd700 1 --2- 172.21.15.43:0/2387695772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9bd8070cb0 0x7f9bd80c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:30.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.198+0000 7f9bd7fff700 1 -- 172.21.15.43:0/2387695772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9bd8070cb0 msgr2=0x7f9bd80c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:30.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.198+0000 7f9bd7fff700 1 --2- 172.21.15.43:0/2387695772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9bd8070cb0 0x7f9bd80c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.198+0000 7f9bd7fff700 1 -- 172.21.15.43:0/2387695772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bd806a600 msgr2=0x7f9bd81f30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:30.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.198+0000 7f9bd7fff700 1 --2- 172.21.15.43:0/2387695772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bd806a600 0x7f9bd81f30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.198+0000 7f9bd7fff700 1 -- 172.21.15.43:0/2387695772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc00094a0 con 0x7f9bd815d9f0 2023-06-28T20:07:30.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.199+0000 7f9bd7fff700 1 --2- 172.21.15.43:0/2387695772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bd815d9f0 0x7f9bd815be30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000cc50 tx=0x7f9bd000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:30.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.199+0000 7f9bcffff700 1 -- 172.21.15.43:0/2387695772 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0003c00 con 0x7f9bd815d9f0 2023-06-28T20:07:30.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.199+0000 7f9bcffff700 1 -- 172.21.15.43:0/2387695772 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd0003d60 con 0x7f9bd815d9f0 2023-06-28T20:07:30.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.199+0000 7f9bdde82700 1 -- 172.21.15.43:0/2387695772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd80c92b0 con 0x7f9bd815d9f0 2023-06-28T20:07:30.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.199+0000 7f9bcffff700 1 -- 172.21.15.43:0/2387695772 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd000aab0 con 0x7f9bd815d9f0 2023-06-28T20:07:30.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.199+0000 7f9bdde82700 1 -- 172.21.15.43:0/2387695772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd80c9870 con 0x7f9bd815d9f0 2023-06-28T20:07:30.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.200+0000 7f9bdde82700 1 -- 172.21.15.43:0/2387695772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd80728b0 con 0x7f9bd815d9f0 2023-06-28T20:07:30.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.200+0000 7f9bcffff700 1 -- 172.21.15.43:0/2387695772 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9bd000ac10 con 0x7f9bd815d9f0 2023-06-28T20:07:30.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.201+0000 7f9bcffff700 1 --2- 172.21.15.43:0/2387695772 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9bb806ace0 0x7f9bb806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:30.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.201+0000 7f9bd77fe700 1 --2- 172.21.15.43:0/2387695772 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9bb806ace0 0x7f9bb806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:30.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.201+0000 7f9bcffff700 1 -- 172.21.15.43:0/2387695772 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9bd00893a0 con 0x7f9bd815d9f0 2023-06-28T20:07:30.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.201+0000 7f9bd77fe700 1 --2- 172.21.15.43:0/2387695772 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9bb806ace0 0x7f9bb806d190 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9bc00090b0 tx=0x7f9bc001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:30.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.205+0000 7f9bcffff700 1 -- 172.21.15.43:0/2387695772 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9bd00544e0 con 0x7f9bd815d9f0 2023-06-28T20:07:30.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.399+0000 7f9bdde82700 1 -- 172.21.15.43:0/2387695772 --> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9bd80cb520 con 0x7f9bb806ace0 2023-06-28T20:07:30.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.407+0000 7f9bcffff700 1 -- 172.21.15.43:0/2387695772 <== mgr.4101 v2:172.21.15.43:6832/406144153 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286909 (secure 0 0 0) 0x7f9bc0011c00 con 0x7f9bb806ace0 2023-06-28T20:07:30.413 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:30.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.411+0000 7f9bdde82700 1 -- 172.21.15.43:0/2387695772 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9bb806ace0 msgr2=0x7f9bb806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:30.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.411+0000 7f9bdde82700 1 --2- 172.21.15.43:0/2387695772 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9bb806ace0 0x7f9bb806d190 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9bc00090b0 tx=0x7f9bc001c040 comp rx=0 tx=0).stop 2023-06-28T20:07:30.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.411+0000 7f9bdde82700 1 -- 172.21.15.43:0/2387695772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bd815d9f0 msgr2=0x7f9bd815be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:30.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.411+0000 7f9bdde82700 1 --2- 172.21.15.43:0/2387695772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bd815d9f0 0x7f9bd815be30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000cc50 tx=0x7f9bd000cf60 comp rx=0 tx=0).stop 2023-06-28T20:07:30.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.411+0000 7f9bdde82700 1 -- 172.21.15.43:0/2387695772 shutdown_connections 2023-06-28T20:07:30.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.411+0000 7f9bdde82700 1 --2- 172.21.15.43:0/2387695772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9bd815d9f0 0x7f9bd815be30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.411+0000 7f9bdde82700 1 --2- 172.21.15.43:0/2387695772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bd806a600 0x7f9bd81f30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.411+0000 7f9bdde82700 1 --2- 172.21.15.43:0/2387695772 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9bb806ace0 0x7f9bb806d190 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.411+0000 7f9bdde82700 1 --2- 172.21.15.43:0/2387695772 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9bd8070cb0 0x7f9bd80c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.411+0000 7f9bdde82700 1 -- 172.21.15.43:0/2387695772 >> 172.21.15.43:0/2387695772 conn(0x7f9bd80b1720 msgr2=0x7f9bd80b2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:30.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.411+0000 7f9bdde82700 1 -- 172.21.15.43:0/2387695772 shutdown_connections 2023-06-28T20:07:30.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.411+0000 7f9bdde82700 1 -- 172.21.15.43:0/2387695772 wait complete. 2023-06-28T20:07:30.417 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-06-28T20:07:30.426 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-06-28T20:07:29.993204+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218532,"kb_used_data":2916,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756188,"statfs":{"total":773094113280,"available":772870336512,"internally_reserved":0,"allocated":2985984,"data_stored":1817291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.631190"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381322+0000","last_change":"2023-06-28T20:07:21.392001+0000","last_active":"2023-06-28T20:07:22.381322+0000","last_peered":"2023-06-28T20:07:22.381322+0000","last_clean":"2023-06-28T20:07:22.381322+0000","last_became_active":"2023-06-28T20:07:21.391521+0000","last_became_peered":"2023-06-28T20:07:21.391521+0000","last_unstale":"2023-06-28T20:07:22.381322+0000","last_undegraded":"2023-06-28T20:07:22.381322+0000","last_fullsized":"2023-06-28T20:07:22.381322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:30:02.751518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787353+0000","last_change":"2023-06-28T20:07:21.381060+0000","last_active":"2023-06-28T20:07:23.787353+0000","last_peered":"2023-06-28T20:07:23.787353+0000","last_clean":"2023-06-28T20:07:23.787353+0000","last_became_active":"2023-06-28T20:07:21.380791+0000","last_became_peered":"2023-06-28T20:07:21.380791+0000","last_unstale":"2023-06-28T20:07:23.787353+0000","last_undegraded":"2023-06-28T20:07:23.787353+0000","last_fullsized":"2023-06-28T20:07:23.787353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:31:11.922720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200155+0000","last_change":"2023-06-28T20:07:21.375652+0000","last_active":"2023-06-28T20:07:23.200155+0000","last_peered":"2023-06-28T20:07:23.200155+0000","last_clean":"2023-06-28T20:07:23.200155+0000","last_became_active":"2023-06-28T20:07:21.375192+0000","last_became_peered":"2023-06-28T20:07:21.375192+0000","last_unstale":"2023-06-28T20:07:23.200155+0000","last_undegraded":"2023-06-28T20:07:23.200155+0000","last_fullsized":"2023-06-28T20:07:23.200155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:38:22.510874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382914+0000","last_change":"2023-06-28T20:07:21.376200+0000","last_active":"2023-06-28T20:07:22.382914+0000","last_peered":"2023-06-28T20:07:22.382914+0000","last_clean":"2023-06-28T20:07:22.382914+0000","last_became_active":"2023-06-28T20:07:21.375927+0000","last_became_peered":"2023-06-28T20:07:21.375927+0000","last_unstale":"2023-06-28T20:07:22.382914+0000","last_undegraded":"2023-06-28T20:07:22.382914+0000","last_fullsized":"2023-06-28T20:07:22.382914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:04:16.061920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786800+0000","last_change":"2023-06-28T20:07:21.394022+0000","last_active":"2023-06-28T20:07:23.786800+0000","last_peered":"2023-06-28T20:07:23.786800+0000","last_clean":"2023-06-28T20:07:23.786800+0000","last_became_active":"2023-06-28T20:07:21.393749+0000","last_became_peered":"2023-06-28T20:07:21.393749+0000","last_unstale":"2023-06-28T20:07:23.786800+0000","last_undegraded":"2023-06-28T20:07:23.786800+0000","last_fullsized":"2023-06-28T20:07:23.786800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:07:49.232842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.199922+0000","last_change":"2023-06-28T20:07:21.380285+0000","last_active":"2023-06-28T20:07:23.199922+0000","last_peered":"2023-06-28T20:07:23.199922+0000","last_clean":"2023-06-28T20:07:23.199922+0000","last_became_active":"2023-06-28T20:07:21.379978+0000","last_became_peered":"2023-06-28T20:07:21.379978+0000","last_unstale":"2023-06-28T20:07:23.199922+0000","last_undegraded":"2023-06-28T20:07:23.199922+0000","last_fullsized":"2023-06-28T20:07:23.199922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:50:32.581801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381458+0000","last_change":"2023-06-28T20:07:21.380983+0000","last_active":"2023-06-28T20:07:22.381458+0000","last_peered":"2023-06-28T20:07:22.381458+0000","last_clean":"2023-06-28T20:07:22.381458+0000","last_became_active":"2023-06-28T20:07:21.380706+0000","last_became_peered":"2023-06-28T20:07:21.380706+0000","last_unstale":"2023-06-28T20:07:22.381458+0000","last_undegraded":"2023-06-28T20:07:22.381458+0000","last_fullsized":"2023-06-28T20:07:22.381458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:43:51.244165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786970+0000","last_change":"2023-06-28T20:07:21.376404+0000","last_active":"2023-06-28T20:07:23.786970+0000","last_peered":"2023-06-28T20:07:23.786970+0000","last_clean":"2023-06-28T20:07:23.786970+0000","last_became_active":"2023-06-28T20:07:21.376136+0000","last_became_peered":"2023-06-28T20:07:21.376136+0000","last_unstale":"2023-06-28T20:07:23.786970+0000","last_undegraded":"2023-06-28T20:07:23.786970+0000","last_fullsized":"2023-06-28T20:07:23.786970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:46:12.205864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787401+0000","last_change":"2023-06-28T20:07:21.394452+0000","last_active":"2023-06-28T20:07:23.787401+0000","last_peered":"2023-06-28T20:07:23.787401+0000","last_clean":"2023-06-28T20:07:23.787401+0000","last_became_active":"2023-06-28T20:07:21.393190+0000","last_became_peered":"2023-06-28T20:07:21.393190+0000","last_unstale":"2023-06-28T20:07:23.787401+0000","last_undegraded":"2023-06-28T20:07:23.787401+0000","last_fullsized":"2023-06-28T20:07:23.787401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:58:50.569412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787514+0000","last_change":"2023-06-28T20:07:21.384870+0000","last_active":"2023-06-28T20:07:23.787514+0000","last_peered":"2023-06-28T20:07:23.787514+0000","last_clean":"2023-06-28T20:07:23.787514+0000","last_became_active":"2023-06-28T20:07:21.384463+0000","last_became_peered":"2023-06-28T20:07:21.384463+0000","last_unstale":"2023-06-28T20:07:23.787514+0000","last_undegraded":"2023-06-28T20:07:23.787514+0000","last_fullsized":"2023-06-28T20:07:23.787514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:44:48.317033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200194+0000","last_change":"2023-06-28T20:07:21.381193+0000","last_active":"2023-06-28T20:07:23.200194+0000","last_peered":"2023-06-28T20:07:23.200194+0000","last_clean":"2023-06-28T20:07:23.200194+0000","last_became_active":"2023-06-28T20:07:21.380917+0000","last_became_peered":"2023-06-28T20:07:21.380917+0000","last_unstale":"2023-06-28T20:07:23.200194+0000","last_undegraded":"2023-06-28T20:07:23.200194+0000","last_fullsized":"2023-06-28T20:07:23.200194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:14:14.828777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381377+0000","last_change":"2023-06-28T20:07:21.382341+0000","last_active":"2023-06-28T20:07:22.381377+0000","last_peered":"2023-06-28T20:07:22.381377+0000","last_clean":"2023-06-28T20:07:22.381377+0000","last_became_active":"2023-06-28T20:07:21.382032+0000","last_became_peered":"2023-06-28T20:07:21.382032+0000","last_unstale":"2023-06-28T20:07:22.381377+0000","last_undegraded":"2023-06-28T20:07:22.381377+0000","last_fullsized":"2023-06-28T20:07:22.381377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T08:00:53.732415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381426+0000","last_change":"2023-06-28T20:07:21.393006+0000","last_active":"2023-06-28T20:07:22.381426+0000","last_peered":"2023-06-28T20:07:22.381426+0000","last_clean":"2023-06-28T20:07:22.381426+0000","last_became_active":"2023-06-28T20:07:21.392710+0000","last_became_peered":"2023-06-28T20:07:21.392710+0000","last_unstale":"2023-06-28T20:07:22.381426+0000","last_undegraded":"2023-06-28T20:07:22.381426+0000","last_fullsized":"2023-06-28T20:07:22.381426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:01:21.282123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787052+0000","last_change":"2023-06-28T20:07:21.380173+0000","last_active":"2023-06-28T20:07:23.787052+0000","last_peered":"2023-06-28T20:07:23.787052+0000","last_clean":"2023-06-28T20:07:23.787052+0000","last_became_active":"2023-06-28T20:07:21.379821+0000","last_became_peered":"2023-06-28T20:07:21.379821+0000","last_unstale":"2023-06-28T20:07:23.787052+0000","last_undegraded":"2023-06-28T20:07:23.787052+0000","last_fullsized":"2023-06-28T20:07:23.787052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:14:35.290431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381515+0000","last_change":"2023-06-28T20:07:21.381063+0000","last_active":"2023-06-28T20:07:22.381515+0000","last_peered":"2023-06-28T20:07:22.381515+0000","last_clean":"2023-06-28T20:07:22.381515+0000","last_became_active":"2023-06-28T20:07:21.380785+0000","last_became_peered":"2023-06-28T20:07:21.380785+0000","last_unstale":"2023-06-28T20:07:22.381515+0000","last_undegraded":"2023-06-28T20:07:22.381515+0000","last_fullsized":"2023-06-28T20:07:22.381515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:48:48.825002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.199680+0000","last_change":"2023-06-28T20:07:21.380326+0000","last_active":"2023-06-28T20:07:23.199680+0000","last_peered":"2023-06-28T20:07:23.199680+0000","last_clean":"2023-06-28T20:07:23.199680+0000","last_became_active":"2023-06-28T20:07:21.380043+0000","last_became_peered":"2023-06-28T20:07:21.380043+0000","last_unstale":"2023-06-28T20:07:23.199680+0000","last_undegraded":"2023-06-28T20:07:23.199680+0000","last_fullsized":"2023-06-28T20:07:23.199680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:27:06.937401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787442+0000","last_change":"2023-06-28T20:07:21.393453+0000","last_active":"2023-06-28T20:07:23.787442+0000","last_peered":"2023-06-28T20:07:23.787442+0000","last_clean":"2023-06-28T20:07:23.787442+0000","last_became_active":"2023-06-28T20:07:21.392856+0000","last_became_peered":"2023-06-28T20:07:21.392856+0000","last_unstale":"2023-06-28T20:07:23.787442+0000","last_undegraded":"2023-06-28T20:07:23.787442+0000","last_fullsized":"2023-06-28T20:07:23.787442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:44:48.317033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200392+0000","last_change":"2023-06-28T20:07:21.387823+0000","last_active":"2023-06-28T20:07:23.200392+0000","last_peered":"2023-06-28T20:07:23.200392+0000","last_clean":"2023-06-28T20:07:23.200392+0000","last_became_active":"2023-06-28T20:07:21.387528+0000","last_became_peered":"2023-06-28T20:07:21.387528+0000","last_unstale":"2023-06-28T20:07:23.200392+0000","last_undegraded":"2023-06-28T20:07:23.200392+0000","last_fullsized":"2023-06-28T20:07:23.200392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:05:11.078362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381385+0000","last_change":"2023-06-28T20:07:21.383174+0000","last_active":"2023-06-28T20:07:22.381385+0000","last_peered":"2023-06-28T20:07:22.381385+0000","last_clean":"2023-06-28T20:07:22.381385+0000","last_became_active":"2023-06-28T20:07:21.382865+0000","last_became_peered":"2023-06-28T20:07:21.382865+0000","last_unstale":"2023-06-28T20:07:22.381385+0000","last_undegraded":"2023-06-28T20:07:22.381385+0000","last_fullsized":"2023-06-28T20:07:22.381385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:19:03.691963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200053+0000","last_change":"2023-06-28T20:07:21.379462+0000","last_active":"2023-06-28T20:07:23.200053+0000","last_peered":"2023-06-28T20:07:23.200053+0000","last_clean":"2023-06-28T20:07:23.200053+0000","last_became_active":"2023-06-28T20:07:21.379190+0000","last_became_peered":"2023-06-28T20:07:21.379190+0000","last_unstale":"2023-06-28T20:07:23.200053+0000","last_undegraded":"2023-06-28T20:07:23.200053+0000","last_fullsized":"2023-06-28T20:07:23.200053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:48:48.825002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381211+0000","last_change":"2023-06-28T20:07:21.392576+0000","last_active":"2023-06-28T20:07:22.381211+0000","last_peered":"2023-06-28T20:07:22.381211+0000","last_clean":"2023-06-28T20:07:22.381211+0000","last_became_active":"2023-06-28T20:07:21.392374+0000","last_became_peered":"2023-06-28T20:07:21.392374+0000","last_unstale":"2023-06-28T20:07:22.381211+0000","last_undegraded":"2023-06-28T20:07:22.381211+0000","last_fullsized":"2023-06-28T20:07:22.381211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:56:31.654337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786384+0000","last_change":"2023-06-28T20:07:21.381495+0000","last_active":"2023-06-28T20:07:23.786384+0000","last_peered":"2023-06-28T20:07:23.786384+0000","last_clean":"2023-06-28T20:07:23.786384+0000","last_became_active":"2023-06-28T20:07:21.380960+0000","last_became_peered":"2023-06-28T20:07:21.380960+0000","last_unstale":"2023-06-28T20:07:23.786384+0000","last_undegraded":"2023-06-28T20:07:23.786384+0000","last_fullsized":"2023-06-28T20:07:23.786384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:16:04.779207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786669+0000","last_change":"2023-06-28T20:07:21.393139+0000","last_active":"2023-06-28T20:07:23.786669+0000","last_peered":"2023-06-28T20:07:23.786669+0000","last_clean":"2023-06-28T20:07:23.786669+0000","last_became_active":"2023-06-28T20:07:21.392866+0000","last_became_peered":"2023-06-28T20:07:21.392866+0000","last_unstale":"2023-06-28T20:07:23.786669+0000","last_undegraded":"2023-06-28T20:07:23.786669+0000","last_fullsized":"2023-06-28T20:07:23.786669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:55:38.748243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.380893+0000","last_change":"2023-06-28T20:07:21.386233+0000","last_active":"2023-06-28T20:07:22.380893+0000","last_peered":"2023-06-28T20:07:22.380893+0000","last_clean":"2023-06-28T20:07:22.380893+0000","last_became_active":"2023-06-28T20:07:21.385374+0000","last_became_peered":"2023-06-28T20:07:21.385374+0000","last_unstale":"2023-06-28T20:07:22.380893+0000","last_undegraded":"2023-06-28T20:07:22.380893+0000","last_fullsized":"2023-06-28T20:07:22.380893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:31:11.922720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.772271+0000","last_change":"2023-06-28T20:07:21.381465+0000","last_active":"2023-06-28T20:07:23.772271+0000","last_peered":"2023-06-28T20:07:23.772271+0000","last_clean":"2023-06-28T20:07:23.772271+0000","last_became_active":"2023-06-28T20:07:21.378500+0000","last_became_peered":"2023-06-28T20:07:21.378500+0000","last_unstale":"2023-06-28T20:07:23.772271+0000","last_undegraded":"2023-06-28T20:07:23.772271+0000","last_fullsized":"2023-06-28T20:07:23.772271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:40:59.613772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.772311+0000","last_change":"2023-06-28T20:07:21.384064+0000","last_active":"2023-06-28T20:07:23.772311+0000","last_peered":"2023-06-28T20:07:23.772311+0000","last_clean":"2023-06-28T20:07:23.772311+0000","last_became_active":"2023-06-28T20:07:21.383797+0000","last_became_peered":"2023-06-28T20:07:21.383797+0000","last_unstale":"2023-06-28T20:07:23.772311+0000","last_undegraded":"2023-06-28T20:07:23.772311+0000","last_fullsized":"2023-06-28T20:07:23.772311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:28:25.317531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787211+0000","last_change":"2023-06-28T20:07:21.382031+0000","last_active":"2023-06-28T20:07:23.787211+0000","last_peered":"2023-06-28T20:07:23.787211+0000","last_clean":"2023-06-28T20:07:23.787211+0000","last_became_active":"2023-06-28T20:07:21.381763+0000","last_became_peered":"2023-06-28T20:07:21.381763+0000","last_unstale":"2023-06-28T20:07:23.787211+0000","last_undegraded":"2023-06-28T20:07:23.787211+0000","last_fullsized":"2023-06-28T20:07:23.787211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:55:38.748243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200003+0000","last_change":"2023-06-28T20:07:20.382468+0000","last_active":"2023-06-28T20:07:23.200003+0000","last_peered":"2023-06-28T20:07:23.200003+0000","last_clean":"2023-06-28T20:07:23.200003+0000","last_became_active":"2023-06-28T20:07:20.382185+0000","last_became_peered":"2023-06-28T20:07:20.382185+0000","last_unstale":"2023-06-28T20:07:23.200003+0000","last_undegraded":"2023-06-28T20:07:23.200003+0000","last_fullsized":"2023-06-28T20:07:23.200003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:43:50.238683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787011+0000","last_change":"2023-06-28T20:07:21.386960+0000","last_active":"2023-06-28T20:07:23.787011+0000","last_peered":"2023-06-28T20:07:23.787011+0000","last_clean":"2023-06-28T20:07:23.787011+0000","last_became_active":"2023-06-28T20:07:21.386705+0000","last_became_peered":"2023-06-28T20:07:21.386705+0000","last_unstale":"2023-06-28T20:07:23.787011+0000","last_undegraded":"2023-06-28T20:07:23.787011+0000","last_fullsized":"2023-06-28T20:07:23.787011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:44:48.317033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.771954+0000","last_change":"2023-06-28T20:07:20.382275+0000","last_active":"2023-06-28T20:07:23.771954+0000","last_peered":"2023-06-28T20:07:23.771954+0000","last_clean":"2023-06-28T20:07:23.771954+0000","last_became_active":"2023-06-28T20:07:20.381923+0000","last_became_peered":"2023-06-28T20:07:20.381923+0000","last_unstale":"2023-06-28T20:07:23.771954+0000","last_undegraded":"2023-06-28T20:07:23.771954+0000","last_fullsized":"2023-06-28T20:07:23.771954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:14:34.284950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787292+0000","last_change":"2023-06-28T20:07:21.380199+0000","last_active":"2023-06-28T20:07:23.787292+0000","last_peered":"2023-06-28T20:07:23.787292+0000","last_clean":"2023-06-28T20:07:23.787292+0000","last_became_active":"2023-06-28T20:07:21.379925+0000","last_became_peered":"2023-06-28T20:07:21.379925+0000","last_unstale":"2023-06-28T20:07:23.787292+0000","last_undegraded":"2023-06-28T20:07:23.787292+0000","last_fullsized":"2023-06-28T20:07:23.787292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:28:25.317531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787110+0000","last_change":"2023-06-28T20:07:20.382465+0000","last_active":"2023-06-28T20:07:23.787110+0000","last_peered":"2023-06-28T20:07:23.787110+0000","last_clean":"2023-06-28T20:07:23.787110+0000","last_became_active":"2023-06-28T20:07:20.380273+0000","last_became_peered":"2023-06-28T20:07:20.380273+0000","last_unstale":"2023-06-28T20:07:23.787110+0000","last_undegraded":"2023-06-28T20:07:23.787110+0000","last_fullsized":"2023-06-28T20:07:23.787110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:27:05.931919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382428+0000","last_change":"2023-06-28T20:07:21.386881+0000","last_active":"2023-06-28T20:07:22.382428+0000","last_peered":"2023-06-28T20:07:22.382428+0000","last_clean":"2023-06-28T20:07:22.382428+0000","last_became_active":"2023-06-28T20:07:21.383395+0000","last_became_peered":"2023-06-28T20:07:21.383395+0000","last_unstale":"2023-06-28T20:07:22.382428+0000","last_undegraded":"2023-06-28T20:07:22.382428+0000","last_fullsized":"2023-06-28T20:07:22.382428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:34:26.678987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787530+0000","last_change":"2023-06-28T20:07:20.378567+0000","last_active":"2023-06-28T20:07:23.787530+0000","last_peered":"2023-06-28T20:07:23.787530+0000","last_clean":"2023-06-28T20:07:23.787530+0000","last_became_active":"2023-06-28T20:07:20.377624+0000","last_became_peered":"2023-06-28T20:07:20.377624+0000","last_unstale":"2023-06-28T20:07:23.787530+0000","last_undegraded":"2023-06-28T20:07:23.787530+0000","last_fullsized":"2023-06-28T20:07:23.787530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:21:31.960769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786981+0000","last_change":"2023-06-28T20:07:21.385250+0000","last_active":"2023-06-28T20:07:23.786981+0000","last_peered":"2023-06-28T20:07:23.786981+0000","last_clean":"2023-06-28T20:07:23.786981+0000","last_became_active":"2023-06-28T20:07:21.384763+0000","last_became_peered":"2023-06-28T20:07:21.384763+0000","last_unstale":"2023-06-28T20:07:23.786981+0000","last_undegraded":"2023-06-28T20:07:23.786981+0000","last_fullsized":"2023-06-28T20:07:23.786981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:58:50.569412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787334+0000","last_change":"2023-06-28T20:07:20.376535+0000","last_active":"2023-06-28T20:07:23.787334+0000","last_peered":"2023-06-28T20:07:23.787334+0000","last_clean":"2023-06-28T20:07:23.787334+0000","last_became_active":"2023-06-28T20:07:20.376182+0000","last_became_peered":"2023-06-28T20:07:20.376182+0000","last_unstale":"2023-06-28T20:07:23.787334+0000","last_undegraded":"2023-06-28T20:07:23.787334+0000","last_fullsized":"2023-06-28T20:07:23.787334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:02:41.673313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787356+0000","last_change":"2023-06-28T20:07:21.390529+0000","last_active":"2023-06-28T20:07:23.787356+0000","last_peered":"2023-06-28T20:07:23.787356+0000","last_clean":"2023-06-28T20:07:23.787356+0000","last_became_active":"2023-06-28T20:07:21.390249+0000","last_became_peered":"2023-06-28T20:07:21.390249+0000","last_unstale":"2023-06-28T20:07:23.787356+0000","last_undegraded":"2023-06-28T20:07:23.787356+0000","last_fullsized":"2023-06-28T20:07:23.787356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:43:51.244165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381357+0000","last_change":"2023-06-28T20:07:20.380921+0000","last_active":"2023-06-28T20:07:22.381357+0000","last_peered":"2023-06-28T20:07:22.381357+0000","last_clean":"2023-06-28T20:07:22.381357+0000","last_became_active":"2023-06-28T20:07:20.380632+0000","last_became_peered":"2023-06-28T20:07:20.380632+0000","last_unstale":"2023-06-28T20:07:22.381357+0000","last_undegraded":"2023-06-28T20:07:22.381357+0000","last_fullsized":"2023-06-28T20:07:22.381357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:03:07.390037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.772038+0000","last_change":"2023-06-28T20:07:21.379317+0000","last_active":"2023-06-28T20:07:23.772038+0000","last_peered":"2023-06-28T20:07:23.772038+0000","last_clean":"2023-06-28T20:07:23.772038+0000","last_became_active":"2023-06-28T20:07:21.379040+0000","last_became_peered":"2023-06-28T20:07:21.379040+0000","last_unstale":"2023-06-28T20:07:23.772038+0000","last_undegraded":"2023-06-28T20:07:23.772038+0000","last_fullsized":"2023-06-28T20:07:23.772038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:04:16.061920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200223+0000","last_change":"2023-06-28T20:07:20.376485+0000","last_active":"2023-06-28T20:07:23.200223+0000","last_peered":"2023-06-28T20:07:23.200223+0000","last_clean":"2023-06-28T20:07:23.200223+0000","last_became_active":"2023-06-28T20:07:20.376240+0000","last_became_peered":"2023-06-28T20:07:20.376240+0000","last_unstale":"2023-06-28T20:07:23.200223+0000","last_undegraded":"2023-06-28T20:07:23.200223+0000","last_fullsized":"2023-06-28T20:07:23.200223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:06:43.811286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200223+0000","last_change":"2023-06-28T20:07:21.379150+0000","last_active":"2023-06-28T20:07:23.200223+0000","last_peered":"2023-06-28T20:07:23.200223+0000","last_clean":"2023-06-28T20:07:23.200223+0000","last_became_active":"2023-06-28T20:07:21.378850+0000","last_became_peered":"2023-06-28T20:07:21.378850+0000","last_unstale":"2023-06-28T20:07:23.200223+0000","last_undegraded":"2023-06-28T20:07:23.200223+0000","last_fullsized":"2023-06-28T20:07:23.200223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:29:17.800360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381032+0000","last_change":"2023-06-28T20:07:20.378538+0000","last_active":"2023-06-28T20:07:22.381032+0000","last_peered":"2023-06-28T20:07:22.381032+0000","last_clean":"2023-06-28T20:07:22.381032+0000","last_became_active":"2023-06-28T20:07:20.378180+0000","last_became_peered":"2023-06-28T20:07:20.378180+0000","last_unstale":"2023-06-28T20:07:22.381032+0000","last_undegraded":"2023-06-28T20:07:22.381032+0000","last_fullsized":"2023-06-28T20:07:22.381032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:58:49.563930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786664+0000","last_change":"2023-06-28T20:07:21.389643+0000","last_active":"2023-06-28T20:07:23.786664+0000","last_peered":"2023-06-28T20:07:23.786664+0000","last_clean":"2023-06-28T20:07:23.786664+0000","last_became_active":"2023-06-28T20:07:21.389163+0000","last_became_peered":"2023-06-28T20:07:21.389163+0000","last_unstale":"2023-06-28T20:07:23.786664+0000","last_undegraded":"2023-06-28T20:07:23.786664+0000","last_fullsized":"2023-06-28T20:07:23.786664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:07:49.232842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786561+0000","last_change":"2023-06-28T20:07:20.371972+0000","last_active":"2023-06-28T20:07:23.786561+0000","last_peered":"2023-06-28T20:07:23.786561+0000","last_clean":"2023-06-28T20:07:23.786561+0000","last_became_active":"2023-06-28T20:07:20.370619+0000","last_became_peered":"2023-06-28T20:07:20.370619+0000","last_unstale":"2023-06-28T20:07:23.786561+0000","last_undegraded":"2023-06-28T20:07:23.786561+0000","last_fullsized":"2023-06-28T20:07:23.786561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:34:25.673506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787272+0000","last_change":"2023-06-28T20:07:21.382142+0000","last_active":"2023-06-28T20:07:23.787272+0000","last_peered":"2023-06-28T20:07:23.787272+0000","last_clean":"2023-06-28T20:07:23.787272+0000","last_became_active":"2023-06-28T20:07:21.381848+0000","last_became_peered":"2023-06-28T20:07:21.381848+0000","last_unstale":"2023-06-28T20:07:23.787272+0000","last_undegraded":"2023-06-28T20:07:23.787272+0000","last_fullsized":"2023-06-28T20:07:23.787272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:40:59.613772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381309+0000","last_change":"2023-06-28T20:07:20.376749+0000","last_active":"2023-06-28T20:07:22.381309+0000","last_peered":"2023-06-28T20:07:22.381309+0000","last_clean":"2023-06-28T20:07:22.381309+0000","last_became_active":"2023-06-28T20:07:20.376465+0000","last_became_peered":"2023-06-28T20:07:20.376465+0000","last_unstale":"2023-06-28T20:07:22.381309+0000","last_undegraded":"2023-06-28T20:07:22.381309+0000","last_fullsized":"2023-06-28T20:07:22.381309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:20:39.309655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787027+0000","last_change":"2023-06-28T20:07:21.393127+0000","last_active":"2023-06-28T20:07:23.787027+0000","last_peered":"2023-06-28T20:07:23.787027+0000","last_clean":"2023-06-28T20:07:23.787027+0000","last_became_active":"2023-06-28T20:07:21.392832+0000","last_became_peered":"2023-06-28T20:07:21.392832+0000","last_unstale":"2023-06-28T20:07:23.787027+0000","last_undegraded":"2023-06-28T20:07:23.787027+0000","last_fullsized":"2023-06-28T20:07:23.787027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:40:59.613772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381838+0000","last_change":"2023-06-28T20:07:20.380053+0000","last_active":"2023-06-28T20:07:22.381838+0000","last_peered":"2023-06-28T20:07:22.381838+0000","last_clean":"2023-06-28T20:07:22.381838+0000","last_became_active":"2023-06-28T20:07:20.379752+0000","last_became_peered":"2023-06-28T20:07:20.379752+0000","last_unstale":"2023-06-28T20:07:22.381838+0000","last_undegraded":"2023-06-28T20:07:22.381838+0000","last_fullsized":"2023-06-28T20:07:22.381838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:58:14.638684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787470+0000","last_change":"2023-06-28T20:07:21.390679+0000","last_active":"2023-06-28T20:07:23.787470+0000","last_peered":"2023-06-28T20:07:23.787470+0000","last_clean":"2023-06-28T20:07:23.787470+0000","last_became_active":"2023-06-28T20:07:21.390332+0000","last_became_peered":"2023-06-28T20:07:21.390332+0000","last_unstale":"2023-06-28T20:07:23.787470+0000","last_undegraded":"2023-06-28T20:07:23.787470+0000","last_fullsized":"2023-06-28T20:07:23.787470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:48:48.825002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381372+0000","last_change":"2023-06-28T20:07:20.379001+0000","last_active":"2023-06-28T20:07:22.381372+0000","last_peered":"2023-06-28T20:07:22.381372+0000","last_clean":"2023-06-28T20:07:22.381372+0000","last_became_active":"2023-06-28T20:07:20.378692+0000","last_became_peered":"2023-06-28T20:07:20.378692+0000","last_unstale":"2023-06-28T20:07:22.381372+0000","last_undegraded":"2023-06-28T20:07:22.381372+0000","last_fullsized":"2023-06-28T20:07:22.381372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:50:31.576319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786912+0000","last_change":"2023-06-28T20:07:21.377599+0000","last_active":"2023-06-28T20:07:23.786912+0000","last_peered":"2023-06-28T20:07:23.786912+0000","last_clean":"2023-06-28T20:07:23.786912+0000","last_became_active":"2023-06-28T20:07:21.377323+0000","last_became_peered":"2023-06-28T20:07:21.377323+0000","last_unstale":"2023-06-28T20:07:23.786912+0000","last_undegraded":"2023-06-28T20:07:23.786912+0000","last_fullsized":"2023-06-28T20:07:23.786912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:34:26.678987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381358+0000","last_change":"2023-06-28T20:07:20.379564+0000","last_active":"2023-06-28T20:07:22.381358+0000","last_peered":"2023-06-28T20:07:22.381358+0000","last_clean":"2023-06-28T20:07:22.381358+0000","last_became_active":"2023-06-28T20:07:20.379179+0000","last_became_peered":"2023-06-28T20:07:20.379179+0000","last_unstale":"2023-06-28T20:07:22.381358+0000","last_undegraded":"2023-06-28T20:07:22.381358+0000","last_fullsized":"2023-06-28T20:07:22.381358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:44:14.718714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.199692+0000","last_change":"2023-06-28T20:07:21.375868+0000","last_active":"2023-06-28T20:07:23.199692+0000","last_peered":"2023-06-28T20:07:23.199692+0000","last_clean":"2023-06-28T20:07:23.199692+0000","last_became_active":"2023-06-28T20:07:21.375273+0000","last_became_peered":"2023-06-28T20:07:21.375273+0000","last_unstale":"2023-06-28T20:07:23.199692+0000","last_undegraded":"2023-06-28T20:07:23.199692+0000","last_fullsized":"2023-06-28T20:07:23.199692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:44:48.317033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381183+0000","last_change":"2023-06-28T20:07:20.377080+0000","last_active":"2023-06-28T20:07:22.381183+0000","last_peered":"2023-06-28T20:07:22.381183+0000","last_clean":"2023-06-28T20:07:22.381183+0000","last_became_active":"2023-06-28T20:07:20.376266+0000","last_became_peered":"2023-06-28T20:07:20.376266+0000","last_unstale":"2023-06-28T20:07:22.381183+0000","last_undegraded":"2023-06-28T20:07:22.381183+0000","last_fullsized":"2023-06-28T20:07:22.381183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:21:31.960769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382189+0000","last_change":"2023-06-28T20:07:21.382463+0000","last_active":"2023-06-28T20:07:22.382189+0000","last_peered":"2023-06-28T20:07:22.382189+0000","last_clean":"2023-06-28T20:07:22.382189+0000","last_became_active":"2023-06-28T20:07:21.382163+0000","last_became_peered":"2023-06-28T20:07:21.382163+0000","last_unstale":"2023-06-28T20:07:22.382189+0000","last_undegraded":"2023-06-28T20:07:22.382189+0000","last_fullsized":"2023-06-28T20:07:22.382189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:44:41.666645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787191+0000","last_change":"2023-06-28T20:07:20.380798+0000","last_active":"2023-06-28T20:07:23.787191+0000","last_peered":"2023-06-28T20:07:23.787191+0000","last_clean":"2023-06-28T20:07:23.787191+0000","last_became_active":"2023-06-28T20:07:20.380380+0000","last_became_peered":"2023-06-28T20:07:20.380380+0000","last_unstale":"2023-06-28T20:07:23.787191+0000","last_undegraded":"2023-06-28T20:07:23.787191+0000","last_fullsized":"2023-06-28T20:07:23.787191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:07:48.227360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786571+0000","last_change":"2023-06-28T20:07:21.382097+0000","last_active":"2023-06-28T20:07:23.786571+0000","last_peered":"2023-06-28T20:07:23.786571+0000","last_clean":"2023-06-28T20:07:23.786571+0000","last_became_active":"2023-06-28T20:07:21.381678+0000","last_became_peered":"2023-06-28T20:07:21.381678+0000","last_unstale":"2023-06-28T20:07:23.786571+0000","last_undegraded":"2023-06-28T20:07:23.786571+0000","last_fullsized":"2023-06-28T20:07:23.786571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:14:35.290431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787297+0000","last_change":"2023-06-28T20:07:20.380519+0000","last_active":"2023-06-28T20:07:23.787297+0000","last_peered":"2023-06-28T20:07:23.787297+0000","last_clean":"2023-06-28T20:07:23.787297+0000","last_became_active":"2023-06-28T20:07:20.380219+0000","last_became_peered":"2023-06-28T20:07:20.380219+0000","last_unstale":"2023-06-28T20:07:23.787297+0000","last_undegraded":"2023-06-28T20:07:23.787297+0000","last_fullsized":"2023-06-28T20:07:23.787297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:50:31.576319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200096+0000","last_change":"2023-06-28T20:07:21.391559+0000","last_active":"2023-06-28T20:07:23.200096+0000","last_peered":"2023-06-28T20:07:23.200096+0000","last_clean":"2023-06-28T20:07:23.200096+0000","last_became_active":"2023-06-28T20:07:21.391214+0000","last_became_peered":"2023-06-28T20:07:21.391214+0000","last_unstale":"2023-06-28T20:07:23.200096+0000","last_undegraded":"2023-06-28T20:07:23.200096+0000","last_fullsized":"2023-06-28T20:07:23.200096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:07:11.795544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382144+0000","last_change":"2023-06-28T20:07:20.378412+0000","last_active":"2023-06-28T20:07:22.382144+0000","last_peered":"2023-06-28T20:07:22.382144+0000","last_clean":"2023-06-28T20:07:22.382144+0000","last_became_active":"2023-06-28T20:07:20.378102+0000","last_became_peered":"2023-06-28T20:07:20.378102+0000","last_unstale":"2023-06-28T20:07:22.382144+0000","last_undegraded":"2023-06-28T20:07:22.382144+0000","last_fullsized":"2023-06-28T20:07:22.382144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:05:45.610772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200420+0000","last_change":"2023-06-28T20:07:21.387960+0000","last_active":"2023-06-28T20:07:23.200420+0000","last_peered":"2023-06-28T20:07:23.200420+0000","last_clean":"2023-06-28T20:07:23.200420+0000","last_became_active":"2023-06-28T20:07:21.387590+0000","last_became_peered":"2023-06-28T20:07:21.387590+0000","last_unstale":"2023-06-28T20:07:23.200420+0000","last_undegraded":"2023-06-28T20:07:23.200420+0000","last_fullsized":"2023-06-28T20:07:23.200420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:06:39.112346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.199929+0000","last_change":"2023-06-28T20:07:20.378036+0000","last_active":"2023-06-28T20:07:23.199929+0000","last_peered":"2023-06-28T20:07:23.199929+0000","last_clean":"2023-06-28T20:07:23.199929+0000","last_became_active":"2023-06-28T20:07:20.377739+0000","last_became_peered":"2023-06-28T20:07:20.377739+0000","last_unstale":"2023-06-28T20:07:23.199929+0000","last_undegraded":"2023-06-28T20:07:23.199929+0000","last_fullsized":"2023-06-28T20:07:23.199929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:44:14.718714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200461+0000","last_change":"2023-06-28T20:07:21.379387+0000","last_active":"2023-06-28T20:07:23.200461+0000","last_peered":"2023-06-28T20:07:23.200461+0000","last_clean":"2023-06-28T20:07:23.200461+0000","last_became_active":"2023-06-28T20:07:21.379090+0000","last_became_peered":"2023-06-28T20:07:21.379090+0000","last_unstale":"2023-06-28T20:07:23.200461+0000","last_undegraded":"2023-06-28T20:07:23.200461+0000","last_fullsized":"2023-06-28T20:07:23.200461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:58:57.345862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786825+0000","last_change":"2023-06-28T20:07:20.382672+0000","last_active":"2023-06-28T20:07:23.786825+0000","last_peered":"2023-06-28T20:07:23.786825+0000","last_clean":"2023-06-28T20:07:23.786825+0000","last_became_active":"2023-06-28T20:07:20.381085+0000","last_became_peered":"2023-06-28T20:07:20.381085+0000","last_unstale":"2023-06-28T20:07:23.786825+0000","last_undegraded":"2023-06-28T20:07:23.786825+0000","last_fullsized":"2023-06-28T20:07:23.786825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:28:24.312050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.772421+0000","last_change":"2023-06-28T20:07:21.386618+0000","last_active":"2023-06-28T20:07:23.772421+0000","last_peered":"2023-06-28T20:07:23.772421+0000","last_clean":"2023-06-28T20:07:23.772421+0000","last_became_active":"2023-06-28T20:07:21.386002+0000","last_became_peered":"2023-06-28T20:07:21.386002+0000","last_unstale":"2023-06-28T20:07:23.772421+0000","last_undegraded":"2023-06-28T20:07:23.772421+0000","last_fullsized":"2023-06-28T20:07:23.772421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:44:48.317033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787225+0000","last_change":"2023-06-28T20:07:20.379096+0000","last_active":"2023-06-28T20:07:23.787225+0000","last_peered":"2023-06-28T20:07:23.787225+0000","last_clean":"2023-06-28T20:07:23.787225+0000","last_became_active":"2023-06-28T20:07:20.378681+0000","last_became_peered":"2023-06-28T20:07:20.378681+0000","last_unstale":"2023-06-28T20:07:23.787225+0000","last_undegraded":"2023-06-28T20:07:23.787225+0000","last_fullsized":"2023-06-28T20:07:23.787225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:03:07.390037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200311+0000","last_change":"2023-06-28T20:07:15.815425+0000","last_active":"2023-06-28T20:07:23.200311+0000","last_peered":"2023-06-28T20:07:23.200311+0000","last_clean":"2023-06-28T20:07:23.200311+0000","last_became_active":"2023-06-28T20:07:15.815163+0000","last_became_peered":"2023-06-28T20:07:15.815163+0000","last_unstale":"2023-06-28T20:07:23.200311+0000","last_undegraded":"2023-06-28T20:07:23.200311+0000","last_fullsized":"2023-06-28T20:07:23.200311+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:50:06.853689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786694+0000","last_change":"2023-06-28T20:07:21.385061+0000","last_active":"2023-06-28T20:07:23.786694+0000","last_peered":"2023-06-28T20:07:23.786694+0000","last_clean":"2023-06-28T20:07:23.786694+0000","last_became_active":"2023-06-28T20:07:21.384404+0000","last_became_peered":"2023-06-28T20:07:21.384404+0000","last_unstale":"2023-06-28T20:07:23.786694+0000","last_undegraded":"2023-06-28T20:07:23.786694+0000","last_fullsized":"2023-06-28T20:07:23.786694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:40:59.613772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786799+0000","last_change":"2023-06-28T20:07:15.303675+0000","last_active":"2023-06-28T20:07:23.786799+0000","last_peered":"2023-06-28T20:07:23.786799+0000","last_clean":"2023-06-28T20:07:23.786799+0000","last_became_active":"2023-06-28T20:07:15.303350+0000","last_became_peered":"2023-06-28T20:07:15.303350+0000","last_unstale":"2023-06-28T20:07:23.786799+0000","last_undegraded":"2023-06-28T20:07:23.786799+0000","last_fullsized":"2023-06-28T20:07:23.786799+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:02:36.611170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"21'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787567+0000","last_change":"2023-06-28T20:07:20.381457+0000","last_active":"2023-06-28T20:07:23.787567+0000","last_peered":"2023-06-28T20:07:23.787567+0000","last_clean":"2023-06-28T20:07:23.787567+0000","last_became_active":"2023-06-28T20:07:20.381235+0000","last_became_peered":"2023-06-28T20:07:20.381235+0000","last_unstale":"2023-06-28T20:07:23.787567+0000","last_undegraded":"2023-06-28T20:07:23.787567+0000","last_fullsized":"2023-06-28T20:07:23.787567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:31:10.917238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.199885+0000","last_change":"2023-06-28T20:07:21.383647+0000","last_active":"2023-06-28T20:07:23.199885+0000","last_peered":"2023-06-28T20:07:23.199885+0000","last_clean":"2023-06-28T20:07:23.199885+0000","last_became_active":"2023-06-28T20:07:21.383392+0000","last_became_peered":"2023-06-28T20:07:21.383392+0000","last_unstale":"2023-06-28T20:07:23.199885+0000","last_undegraded":"2023-06-28T20:07:23.199885+0000","last_fullsized":"2023-06-28T20:07:23.199885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:03:08.395518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.771713+0000","last_change":"2023-06-28T20:07:15.303565+0000","last_active":"2023-06-28T20:07:23.771713+0000","last_peered":"2023-06-28T20:07:23.771713+0000","last_clean":"2023-06-28T20:07:23.771713+0000","last_became_active":"2023-06-28T20:07:15.303200+0000","last_became_peered":"2023-06-28T20:07:15.303200+0000","last_unstale":"2023-06-28T20:07:23.771713+0000","last_undegraded":"2023-06-28T20:07:23.771713+0000","last_fullsized":"2023-06-28T20:07:23.771713+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:46:06.138239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":169,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381052+0000","last_change":"2023-06-28T20:07:15.304921+0000","last_active":"2023-06-28T20:07:22.381052+0000","last_peered":"2023-06-28T20:07:22.381052+0000","last_clean":"2023-06-28T20:07:22.381052+0000","last_became_active":"2023-06-28T20:07:15.303959+0000","last_became_peered":"2023-06-28T20:07:15.303959+0000","last_unstale":"2023-06-28T20:07:22.381052+0000","last_undegraded":"2023-06-28T20:07:22.381052+0000","last_fullsized":"2023-06-28T20:07:22.381052+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:44:42.249408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786530+0000","last_change":"2023-06-28T20:07:20.379375+0000","last_active":"2023-06-28T20:07:23.786530+0000","last_peered":"2023-06-28T20:07:23.786530+0000","last_clean":"2023-06-28T20:07:23.786530+0000","last_became_active":"2023-06-28T20:07:20.378992+0000","last_became_peered":"2023-06-28T20:07:20.378992+0000","last_unstale":"2023-06-28T20:07:23.786530+0000","last_undegraded":"2023-06-28T20:07:23.786530+0000","last_fullsized":"2023-06-28T20:07:23.786530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:46:11.200383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787157+0000","last_change":"2023-06-28T20:07:21.383270+0000","last_active":"2023-06-28T20:07:23.787157+0000","last_peered":"2023-06-28T20:07:23.787157+0000","last_clean":"2023-06-28T20:07:23.787157+0000","last_became_active":"2023-06-28T20:07:21.382983+0000","last_became_peered":"2023-06-28T20:07:21.382983+0000","last_unstale":"2023-06-28T20:07:23.787157+0000","last_undegraded":"2023-06-28T20:07:23.787157+0000","last_fullsized":"2023-06-28T20:07:23.787157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:02:42.678795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.380861+0000","last_change":"2023-06-28T20:07:15.899048+0000","last_active":"2023-06-28T20:07:22.380861+0000","last_peered":"2023-06-28T20:07:22.380861+0000","last_clean":"2023-06-28T20:07:22.380861+0000","last_became_active":"2023-06-28T20:07:15.898766+0000","last_became_peered":"2023-06-28T20:07:15.898766+0000","last_unstale":"2023-06-28T20:07:22.380861+0000","last_undegraded":"2023-06-28T20:07:22.380861+0000","last_fullsized":"2023-06-28T20:07:22.380861+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:27:00.869776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.771898+0000","last_change":"2023-06-28T20:07:20.369864+0000","last_active":"2023-06-28T20:07:23.771898+0000","last_peered":"2023-06-28T20:07:23.771898+0000","last_clean":"2023-06-28T20:07:23.771898+0000","last_became_active":"2023-06-28T20:07:20.369542+0000","last_became_peered":"2023-06-28T20:07:20.369542+0000","last_unstale":"2023-06-28T20:07:23.771898+0000","last_undegraded":"2023-06-28T20:07:23.771898+0000","last_fullsized":"2023-06-28T20:07:23.771898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:34:25.673506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.772302+0000","last_change":"2023-06-28T20:07:21.383019+0000","last_active":"2023-06-28T20:07:23.772302+0000","last_peered":"2023-06-28T20:07:23.772302+0000","last_clean":"2023-06-28T20:07:23.772302+0000","last_became_active":"2023-06-28T20:07:21.382756+0000","last_became_peered":"2023-06-28T20:07:21.382756+0000","last_unstale":"2023-06-28T20:07:23.772302+0000","last_undegraded":"2023-06-28T20:07:23.772302+0000","last_fullsized":"2023-06-28T20:07:23.772302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:02:42.678795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.380789+0000","last_change":"2023-06-28T20:07:15.899229+0000","last_active":"2023-06-28T20:07:22.380789+0000","last_peered":"2023-06-28T20:07:22.380789+0000","last_clean":"2023-06-28T20:07:22.380789+0000","last_became_active":"2023-06-28T20:07:15.898708+0000","last_became_peered":"2023-06-28T20:07:15.898708+0000","last_unstale":"2023-06-28T20:07:22.380789+0000","last_undegraded":"2023-06-28T20:07:22.380789+0000","last_fullsized":"2023-06-28T20:07:22.380789+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:28:19.249906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787350+0000","last_change":"2023-06-28T20:07:20.381183+0000","last_active":"2023-06-28T20:07:23.787350+0000","last_peered":"2023-06-28T20:07:23.787350+0000","last_clean":"2023-06-28T20:07:23.787350+0000","last_became_active":"2023-06-28T20:07:20.380859+0000","last_became_peered":"2023-06-28T20:07:20.380859+0000","last_unstale":"2023-06-28T20:07:23.787350+0000","last_undegraded":"2023-06-28T20:07:23.787350+0000","last_fullsized":"2023-06-28T20:07:23.787350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:44:14.718714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382351+0000","last_change":"2023-06-28T20:07:21.382900+0000","last_active":"2023-06-28T20:07:22.382351+0000","last_peered":"2023-06-28T20:07:22.382351+0000","last_clean":"2023-06-28T20:07:22.382351+0000","last_became_active":"2023-06-28T20:07:21.382384+0000","last_became_peered":"2023-06-28T20:07:21.382384+0000","last_unstale":"2023-06-28T20:07:22.382351+0000","last_undegraded":"2023-06-28T20:07:22.382351+0000","last_fullsized":"2023-06-28T20:07:22.382351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:16:04.779207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786990+0000","last_change":"2023-06-28T20:07:15.898966+0000","last_active":"2023-06-28T20:07:23.786990+0000","last_peered":"2023-06-28T20:07:23.786990+0000","last_clean":"2023-06-28T20:07:23.786990+0000","last_became_active":"2023-06-28T20:07:15.898704+0000","last_became_peered":"2023-06-28T20:07:15.898704+0000","last_unstale":"2023-06-28T20:07:23.786990+0000","last_undegraded":"2023-06-28T20:07:23.786990+0000","last_fullsized":"2023-06-28T20:07:23.786990+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:15:58.711582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.199867+0000","last_change":"2023-06-28T20:07:20.378848+0000","last_active":"2023-06-28T20:07:23.199867+0000","last_peered":"2023-06-28T20:07:23.199867+0000","last_clean":"2023-06-28T20:07:23.199867+0000","last_became_active":"2023-06-28T20:07:20.378438+0000","last_became_peered":"2023-06-28T20:07:20.378438+0000","last_unstale":"2023-06-28T20:07:23.199867+0000","last_undegraded":"2023-06-28T20:07:23.199867+0000","last_fullsized":"2023-06-28T20:07:23.199867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:20:39.309655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.772343+0000","last_change":"2023-06-28T20:07:21.384171+0000","last_active":"2023-06-28T20:07:23.772343+0000","last_peered":"2023-06-28T20:07:23.772343+0000","last_clean":"2023-06-28T20:07:23.772343+0000","last_became_active":"2023-06-28T20:07:21.383867+0000","last_became_peered":"2023-06-28T20:07:21.383867+0000","last_unstale":"2023-06-28T20:07:23.772343+0000","last_undegraded":"2023-06-28T20:07:23.772343+0000","last_fullsized":"2023-06-28T20:07:23.772343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:58:50.569412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382412+0000","last_change":"2023-06-28T20:07:15.899207+0000","last_active":"2023-06-28T20:07:22.382412+0000","last_peered":"2023-06-28T20:07:22.382412+0000","last_clean":"2023-06-28T20:07:22.382412+0000","last_became_active":"2023-06-28T20:07:15.898908+0000","last_became_peered":"2023-06-28T20:07:15.898908+0000","last_unstale":"2023-06-28T20:07:22.382412+0000","last_undegraded":"2023-06-28T20:07:22.382412+0000","last_fullsized":"2023-06-28T20:07:22.382412+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:14:29.222806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.772318+0000","last_change":"2023-06-28T20:07:20.377530+0000","last_active":"2023-06-28T20:07:23.772318+0000","last_peered":"2023-06-28T20:07:23.772318+0000","last_clean":"2023-06-28T20:07:23.772318+0000","last_became_active":"2023-06-28T20:07:20.377243+0000","last_became_peered":"2023-06-28T20:07:20.377243+0000","last_unstale":"2023-06-28T20:07:23.772318+0000","last_undegraded":"2023-06-28T20:07:23.772318+0000","last_fullsized":"2023-06-28T20:07:23.772318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:31:10.917238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786957+0000","last_change":"2023-06-28T20:07:21.383886+0000","last_active":"2023-06-28T20:07:23.786957+0000","last_peered":"2023-06-28T20:07:23.786957+0000","last_clean":"2023-06-28T20:07:23.786957+0000","last_became_active":"2023-06-28T20:07:21.383572+0000","last_became_peered":"2023-06-28T20:07:21.383572+0000","last_unstale":"2023-06-28T20:07:23.786957+0000","last_undegraded":"2023-06-28T20:07:23.786957+0000","last_fullsized":"2023-06-28T20:07:23.786957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:55:38.748243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382342+0000","last_change":"2023-06-28T20:07:15.303873+0000","last_active":"2023-06-28T20:07:22.382342+0000","last_peered":"2023-06-28T20:07:22.382342+0000","last_clean":"2023-06-28T20:07:22.382342+0000","last_became_active":"2023-06-28T20:07:15.303457+0000","last_became_peered":"2023-06-28T20:07:15.303457+0000","last_unstale":"2023-06-28T20:07:22.382342+0000","last_undegraded":"2023-06-28T20:07:22.382342+0000","last_fullsized":"2023-06-28T20:07:22.382342+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:46:06.138239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787461+0000","last_change":"2023-06-28T20:07:20.381529+0000","last_active":"2023-06-28T20:07:23.787461+0000","last_peered":"2023-06-28T20:07:23.787461+0000","last_clean":"2023-06-28T20:07:23.787461+0000","last_became_active":"2023-06-28T20:07:20.381191+0000","last_became_peered":"2023-06-28T20:07:20.381191+0000","last_unstale":"2023-06-28T20:07:23.787461+0000","last_undegraded":"2023-06-28T20:07:23.787461+0000","last_fullsized":"2023-06-28T20:07:23.787461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:58:49.563930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.772112+0000","last_change":"2023-06-28T20:07:21.377519+0000","last_active":"2023-06-28T20:07:23.772112+0000","last_peered":"2023-06-28T20:07:23.772112+0000","last_clean":"2023-06-28T20:07:23.772112+0000","last_became_active":"2023-06-28T20:07:21.377225+0000","last_became_peered":"2023-06-28T20:07:21.377225+0000","last_unstale":"2023-06-28T20:07:23.772112+0000","last_undegraded":"2023-06-28T20:07:23.772112+0000","last_fullsized":"2023-06-28T20:07:23.772112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:07:49.232842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786710+0000","last_change":"2023-06-28T20:07:20.382678+0000","last_active":"2023-06-28T20:07:23.786710+0000","last_peered":"2023-06-28T20:07:23.786710+0000","last_clean":"2023-06-28T20:07:23.786710+0000","last_became_active":"2023-06-28T20:07:20.382361+0000","last_became_peered":"2023-06-28T20:07:20.382361+0000","last_unstale":"2023-06-28T20:07:23.786710+0000","last_undegraded":"2023-06-28T20:07:23.786710+0000","last_fullsized":"2023-06-28T20:07:23.786710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:04:15.056439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381381+0000","last_change":"2023-06-28T20:07:21.397232+0000","last_active":"2023-06-28T20:07:22.381381+0000","last_peered":"2023-06-28T20:07:22.381381+0000","last_clean":"2023-06-28T20:07:22.381381+0000","last_became_active":"2023-06-28T20:07:21.396862+0000","last_became_peered":"2023-06-28T20:07:21.396862+0000","last_unstale":"2023-06-28T20:07:22.381381+0000","last_undegraded":"2023-06-28T20:07:22.381381+0000","last_fullsized":"2023-06-28T20:07:22.381381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:48:55.783926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787099+0000","last_change":"2023-06-28T20:07:20.380628+0000","last_active":"2023-06-28T20:07:23.787099+0000","last_peered":"2023-06-28T20:07:23.787099+0000","last_clean":"2023-06-28T20:07:23.787099+0000","last_became_active":"2023-06-28T20:07:20.380033+0000","last_became_peered":"2023-06-28T20:07:20.380033+0000","last_unstale":"2023-06-28T20:07:23.787099+0000","last_undegraded":"2023-06-28T20:07:23.787099+0000","last_fullsized":"2023-06-28T20:07:23.787099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:04:15.056439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.199812+0000","last_change":"2023-06-28T20:07:21.374666+0000","last_active":"2023-06-28T20:07:23.199812+0000","last_peered":"2023-06-28T20:07:23.199812+0000","last_clean":"2023-06-28T20:07:23.199812+0000","last_became_active":"2023-06-28T20:07:21.374330+0000","last_became_peered":"2023-06-28T20:07:21.374330+0000","last_unstale":"2023-06-28T20:07:23.199812+0000","last_undegraded":"2023-06-28T20:07:23.199812+0000","last_fullsized":"2023-06-28T20:07:23.199812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:21:32.966251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787460+0000","last_change":"2023-06-28T20:07:20.378725+0000","last_active":"2023-06-28T20:07:23.787460+0000","last_peered":"2023-06-28T20:07:23.787460+0000","last_clean":"2023-06-28T20:07:23.787460+0000","last_became_active":"2023-06-28T20:07:20.378445+0000","last_became_peered":"2023-06-28T20:07:20.378445+0000","last_unstale":"2023-06-28T20:07:23.787460+0000","last_undegraded":"2023-06-28T20:07:23.787460+0000","last_fullsized":"2023-06-28T20:07:23.787460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:28:24.312050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200081+0000","last_change":"2023-06-28T20:07:21.374603+0000","last_active":"2023-06-28T20:07:23.200081+0000","last_peered":"2023-06-28T20:07:23.200081+0000","last_clean":"2023-06-28T20:07:23.200081+0000","last_became_active":"2023-06-28T20:07:21.374276+0000","last_became_peered":"2023-06-28T20:07:21.374276+0000","last_unstale":"2023-06-28T20:07:23.200081+0000","last_undegraded":"2023-06-28T20:07:23.200081+0000","last_fullsized":"2023-06-28T20:07:23.200081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:42:24.558661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787168+0000","last_change":"2023-06-28T20:07:20.382290+0000","last_active":"2023-06-28T20:07:23.787168+0000","last_peered":"2023-06-28T20:07:23.787168+0000","last_clean":"2023-06-28T20:07:23.787168+0000","last_became_active":"2023-06-28T20:07:20.381991+0000","last_became_peered":"2023-06-28T20:07:20.381991+0000","last_unstale":"2023-06-28T20:07:23.787168+0000","last_undegraded":"2023-06-28T20:07:23.787168+0000","last_fullsized":"2023-06-28T20:07:23.787168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:56:30.648855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786827+0000","last_change":"2023-06-28T20:07:21.391299+0000","last_active":"2023-06-28T20:07:23.786827+0000","last_peered":"2023-06-28T20:07:23.786827+0000","last_clean":"2023-06-28T20:07:23.786827+0000","last_became_active":"2023-06-28T20:07:21.390957+0000","last_became_peered":"2023-06-28T20:07:21.390957+0000","last_unstale":"2023-06-28T20:07:23.786827+0000","last_undegraded":"2023-06-28T20:07:23.786827+0000","last_fullsized":"2023-06-28T20:07:23.786827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:20:40.315136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787523+0000","last_change":"2023-06-28T20:07:20.378108+0000","last_active":"2023-06-28T20:07:23.787523+0000","last_peered":"2023-06-28T20:07:23.787523+0000","last_clean":"2023-06-28T20:07:23.787523+0000","last_became_active":"2023-06-28T20:07:20.377812+0000","last_became_peered":"2023-06-28T20:07:20.377812+0000","last_unstale":"2023-06-28T20:07:23.787523+0000","last_undegraded":"2023-06-28T20:07:23.787523+0000","last_fullsized":"2023-06-28T20:07:23.787523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:27:05.931919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.199602+0000","last_change":"2023-06-28T20:07:21.384739+0000","last_active":"2023-06-28T20:07:23.199602+0000","last_peered":"2023-06-28T20:07:23.199602+0000","last_clean":"2023-06-28T20:07:23.199602+0000","last_became_active":"2023-06-28T20:07:21.384136+0000","last_became_peered":"2023-06-28T20:07:21.384136+0000","last_unstale":"2023-06-28T20:07:23.199602+0000","last_undegraded":"2023-06-28T20:07:23.199602+0000","last_fullsized":"2023-06-28T20:07:23.199602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:58:50.569412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382228+0000","last_change":"2023-06-28T20:07:21.386188+0000","last_active":"2023-06-28T20:07:22.382228+0000","last_peered":"2023-06-28T20:07:22.382228+0000","last_clean":"2023-06-28T20:07:22.382228+0000","last_became_active":"2023-06-28T20:07:21.385171+0000","last_became_peered":"2023-06-28T20:07:21.385171+0000","last_unstale":"2023-06-28T20:07:22.382228+0000","last_undegraded":"2023-06-28T20:07:22.382228+0000","last_fullsized":"2023-06-28T20:07:22.382228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:43:18.447196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786806+0000","last_change":"2023-06-28T20:07:21.402344+0000","last_active":"2023-06-28T20:07:23.786806+0000","last_peered":"2023-06-28T20:07:23.786806+0000","last_clean":"2023-06-28T20:07:23.786806+0000","last_became_active":"2023-06-28T20:07:21.402005+0000","last_became_peered":"2023-06-28T20:07:21.402005+0000","last_unstale":"2023-06-28T20:07:23.786806+0000","last_undegraded":"2023-06-28T20:07:23.786806+0000","last_fullsized":"2023-06-28T20:07:23.786806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:31:11.922720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.199807+0000","last_change":"2023-06-28T20:07:21.384643+0000","last_active":"2023-06-28T20:07:23.199807+0000","last_peered":"2023-06-28T20:07:23.199807+0000","last_clean":"2023-06-28T20:07:23.199807+0000","last_became_active":"2023-06-28T20:07:21.384232+0000","last_became_peered":"2023-06-28T20:07:21.384232+0000","last_unstale":"2023-06-28T20:07:23.199807+0000","last_undegraded":"2023-06-28T20:07:23.199807+0000","last_fullsized":"2023-06-28T20:07:23.199807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:56:31.654337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787158+0000","last_change":"2023-06-28T20:07:21.391594+0000","last_active":"2023-06-28T20:07:23.787158+0000","last_peered":"2023-06-28T20:07:23.787158+0000","last_clean":"2023-06-28T20:07:23.787158+0000","last_became_active":"2023-06-28T20:07:21.391222+0000","last_became_peered":"2023-06-28T20:07:21.391222+0000","last_unstale":"2023-06-28T20:07:23.787158+0000","last_undegraded":"2023-06-28T20:07:23.787158+0000","last_fullsized":"2023-06-28T20:07:23.787158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:21:32.966251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.772192+0000","last_change":"2023-06-28T20:07:21.378640+0000","last_active":"2023-06-28T20:07:23.772192+0000","last_peered":"2023-06-28T20:07:23.772192+0000","last_clean":"2023-06-28T20:07:23.772192+0000","last_became_active":"2023-06-28T20:07:21.378345+0000","last_became_peered":"2023-06-28T20:07:21.378345+0000","last_unstale":"2023-06-28T20:07:23.772192+0000","last_undegraded":"2023-06-28T20:07:23.772192+0000","last_fullsized":"2023-06-28T20:07:23.772192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:55:38.748243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":839680,"data_stored":671247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":79099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":811008,"data_stored":669467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":78720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":79099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":221184,"data_stored":79189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":80719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.0700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":225280,"data_stored":79751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-28T20:07:30.432 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-28T20:07:30.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.587+0000 7ff7984d5700 1 Processor -- start 2023-06-28T20:07:30.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.587+0000 7ff7984d5700 1 -- start start 2023-06-28T20:07:30.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff7984d5700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7900ce170 0x7ff7900c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:30.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff7984d5700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff79015a960 con 0x7ff7900ce870 2023-06-28T20:07:30.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff7984d5700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7900ca490 con 0x7ff79014ff80 2023-06-28T20:07:30.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff7984d5700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7900ca770 con 0x7ff7900ce170 2023-06-28T20:07:30.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff796a72700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff7900ce870 0x7ff790069ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42566/0 (socket says 172.21.15.43:42566) 2023-06-28T20:07:30.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff796a72700 1 -- 172.21.15.43:0/2337858590 learned_addr learned my addr 172.21.15.43:0/2337858590 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:30.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff795a70700 1 --2- 172.21.15.43:0/2337858590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7900ce170 0x7ff7900c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:30.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff79526f700 1 -- 172.21.15.43:0/2337858590 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890441218 0 0) 0x7ff79015a960 con 0x7ff7900ce870 2023-06-28T20:07:30.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff79526f700 1 -- 172.21.15.43:0/2337858590 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff77c0035b0 con 0x7ff7900ce870 2023-06-28T20:07:30.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff79526f700 1 -- 172.21.15.43:0/2337858590 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 148935314 0 0) 0x7ff7900ca490 con 0x7ff79014ff80 2023-06-28T20:07:30.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff79526f700 1 -- 172.21.15.43:0/2337858590 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff79015a960 con 0x7ff79014ff80 2023-06-28T20:07:30.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff79526f700 1 -- 172.21.15.43:0/2337858590 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1382896702 0 0) 0x7ff77c0035b0 con 0x7ff7900ce870 2023-06-28T20:07:30.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff79526f700 1 -- 172.21.15.43:0/2337858590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7900ce170 msgr2=0x7ff7900c9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:30.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff79526f700 1 --2- 172.21.15.43:0/2337858590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7900ce170 0x7ff7900c9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff79526f700 1 -- 172.21.15.43:0/2337858590 >> v1:172.21.15.178:6789/0 conn(0x7ff79014ff80 legacy=0x7ff790150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:30.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff79526f700 1 -- 172.21.15.43:0/2337858590 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7900cd6b0 con 0x7ff7900ce870 2023-06-28T20:07:30.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff79526f700 1 -- 172.21.15.43:0/2337858590 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7ff7880019f0 con 0x7ff7900ce870 2023-06-28T20:07:30.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff79526f700 1 -- 172.21.15.43:0/2337858590 >> v1:172.21.15.43:6789/0 conn(0x7ff7900ce870 legacy=0x7ff790069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:30.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff79526f700 1 --2- 172.21.15.43:0/2337858590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7900ce170 0x7ff77c003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:30.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff79526f700 1 --2- 172.21.15.43:0/2337858590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff77c004120 0x7ff77c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:30.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff79526f700 1 --2- 172.21.15.43:0/2337858590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff77c004d90 0x7ff77c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:30.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff79526f700 1 -- 172.21.15.43:0/2337858590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7900ca770 con 0x7ff77c004120 2023-06-28T20:07:30.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff79526f700 1 -- 172.21.15.43:0/2337858590 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c00bac0 con 0x7ff77c004d90 2023-06-28T20:07:30.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff79526f700 1 -- 172.21.15.43:0/2337858590 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c00bc80 con 0x7ff7900ce170 2023-06-28T20:07:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.588+0000 7ff795a70700 1 --2- 172.21.15.43:0/2337858590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff77c004120 0x7ff77c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.589+0000 7ff796271700 1 --2- 172.21.15.43:0/2337858590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7900ce170 0x7ff77c003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.589+0000 7ff796a72700 1 --2- 172.21.15.43:0/2337858590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff77c004d90 0x7ff77c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.589+0000 7ff795a70700 1 -- 172.21.15.43:0/2337858590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7900ce170 msgr2=0x7ff77c003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.589+0000 7ff795a70700 1 --2- 172.21.15.43:0/2337858590 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff7900ce170 0x7ff77c003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.589+0000 7ff795a70700 1 -- 172.21.15.43:0/2337858590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff77c004d90 msgr2=0x7ff77c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.589+0000 7ff795a70700 1 --2- 172.21.15.43:0/2337858590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff77c004d90 0x7ff77c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.589+0000 7ff795a70700 1 -- 172.21.15.43:0/2337858590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77c0035b0 con 0x7ff77c004120 2023-06-28T20:07:30.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.589+0000 7ff795a70700 1 --2- 172.21.15.43:0/2337858590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff77c004120 0x7ff77c004850 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff7800096f0 tx=0x7ff77c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c4f0705ff9d488 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:30.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.589+0000 7ff79526f700 1 -- 172.21.15.43:0/2337858590 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78000e6c0 con 0x7ff77c004120 2023-06-28T20:07:30.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.589+0000 7ff79526f700 1 -- 172.21.15.43:0/2337858590 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78000e820 con 0x7ff77c004120 2023-06-28T20:07:30.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.589+0000 7ff79526f700 1 -- 172.21.15.43:0/2337858590 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78000ebb0 con 0x7ff77c004120 2023-06-28T20:07:30.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.590+0000 7ff7984d5700 1 -- 172.21.15.43:0/2337858590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff77c004120 msgr2=0x7ff77c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:30.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.590+0000 7ff7984d5700 1 --2- 172.21.15.43:0/2337858590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff77c004120 0x7ff77c004850 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff7800096f0 tx=0x7ff77c005880 comp rx=0 tx=0).stop 2023-06-28T20:07:30.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.590+0000 7ff796a72700 1 -- 172.21.15.43:0/2337858590 reap_dead start 2023-06-28T20:07:30.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.590+0000 7ff7984d5700 1 -- 172.21.15.43:0/2337858590 shutdown_connections 2023-06-28T20:07:30.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.590+0000 7ff7984d5700 1 -- 172.21.15.43:0/2337858590 >> 172.21.15.43:0/2337858590 conn(0x7ff79006c170 msgr2=0x7ff79006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:30.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.590+0000 7ff7984d5700 1 -- 172.21.15.43:0/2337858590 shutdown_connections 2023-06-28T20:07:30.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.590+0000 7ff7984d5700 1 -- 172.21.15.43:0/2337858590 wait complete. 2023-06-28T20:07:30.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.591+0000 7ff7984d5700 1 Processor -- start 2023-06-28T20:07:30.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.591+0000 7ff7984d5700 1 -- start start 2023-06-28T20:07:30.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.592+0000 7ff7984d5700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff77c004120 0x7ff7900c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:30.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.592+0000 7ff7984d5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7901edaf0 0x7ff7900c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:30.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.592+0000 7ff7984d5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7900c73a0 0x7ff7901589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:30.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.592+0000 7ff7984d5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7901eeba0 con 0x7ff7901edaf0 2023-06-28T20:07:30.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.592+0000 7ff7984d5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7901edf90 con 0x7ff7900c73a0 2023-06-28T20:07:30.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.592+0000 7ff7984d5700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7901ee270 con 0x7ff77c004120 2023-06-28T20:07:30.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.592+0000 7ff795a70700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7901edaf0 0x7ff7900c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:30.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.592+0000 7ff795a70700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7901edaf0 0x7ff7900c6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53390/0 (socket says 172.21.15.43:53390) 2023-06-28T20:07:30.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.592+0000 7ff795a70700 1 -- 172.21.15.43:0/1882398532 learned_addr learned my addr 172.21.15.43:0/1882398532 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:30.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.593+0000 7ff796271700 1 --2- 172.21.15.43:0/1882398532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff77c004120 0x7ff7900c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:30.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.593+0000 7ff796a72700 1 --2- 172.21.15.43:0/1882398532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7900c73a0 0x7ff7901589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:30.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.593+0000 7ff795a70700 1 -- 172.21.15.43:0/1882398532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff77c004120 msgr2=0x7ff7900c53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:30.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.593+0000 7ff795a70700 1 --2- 172.21.15.43:0/1882398532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff77c004120 0x7ff7900c53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.593+0000 7ff795a70700 1 -- 172.21.15.43:0/1882398532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7900c73a0 msgr2=0x7ff7901589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:30.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.593+0000 7ff795a70700 1 --2- 172.21.15.43:0/1882398532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7900c73a0 0x7ff7901589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.593+0000 7ff795a70700 1 -- 172.21.15.43:0/1882398532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7901ee550 con 0x7ff7901edaf0 2023-06-28T20:07:30.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.593+0000 7ff795a70700 1 --2- 172.21.15.43:0/1882398532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7901edaf0 0x7ff7900c6e60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff7800090b0 tx=0x7ff7800047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:30.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.593+0000 7ff786ffd700 1 -- 172.21.15.43:0/1882398532 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78000e6c0 con 0x7ff7901edaf0 2023-06-28T20:07:30.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.593+0000 7ff786ffd700 1 -- 172.21.15.43:0/1882398532 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7800038c0 con 0x7ff7901edaf0 2023-06-28T20:07:30.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.593+0000 7ff7984d5700 1 -- 172.21.15.43:0/1882398532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7901ee830 con 0x7ff7901edaf0 2023-06-28T20:07:30.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.593+0000 7ff786ffd700 1 -- 172.21.15.43:0/1882398532 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff780006e60 con 0x7ff7901edaf0 2023-06-28T20:07:30.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.593+0000 7ff7984d5700 1 -- 172.21.15.43:0/1882398532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7901590e0 con 0x7ff7901edaf0 2023-06-28T20:07:30.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.594+0000 7ff7984d5700 1 -- 172.21.15.43:0/1882398532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff790159300 con 0x7ff7901edaf0 2023-06-28T20:07:30.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.595+0000 7ff786ffd700 1 -- 172.21.15.43:0/1882398532 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff780020040 con 0x7ff7901edaf0 2023-06-28T20:07:30.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.595+0000 7ff786ffd700 1 --2- 172.21.15.43:0/1882398532 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff77406ace0 0x7ff77406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:30.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.595+0000 7ff786ffd700 1 -- 172.21.15.43:0/1882398532 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff78008b9d0 con 0x7ff7901edaf0 2023-06-28T20:07:30.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.595+0000 7ff796271700 1 --2- 172.21.15.43:0/1882398532 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff77406ace0 0x7ff77406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:30.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.596+0000 7ff796271700 1 --2- 172.21.15.43:0/1882398532 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff77406ace0 0x7ff77406d190 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff7901eb730 tx=0x7ff78c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:30.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.599+0000 7ff786ffd700 1 -- 172.21.15.43:0/1882398532 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff780056b10 con 0x7ff7901edaf0 2023-06-28T20:07:30.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.794+0000 7ff7984d5700 1 -- 172.21.15.43:0/1882398532 --> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff7900cdc70 con 0x7ff77406ace0 2023-06-28T20:07:30.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.802+0000 7ff786ffd700 1 -- 172.21.15.43:0/1882398532 <== mgr.4101 v2:172.21.15.43:6832/406144153 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286909 (secure 0 0 0) 0x7ff78c001730 con 0x7ff77406ace0 2023-06-28T20:07:30.809 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:30.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.806+0000 7ff7984d5700 1 -- 172.21.15.43:0/1882398532 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff77406ace0 msgr2=0x7ff77406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:30.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.806+0000 7ff7984d5700 1 --2- 172.21.15.43:0/1882398532 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff77406ace0 0x7ff77406d190 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff7901eb730 tx=0x7ff78c009040 comp rx=0 tx=0).stop 2023-06-28T20:07:30.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.806+0000 7ff7984d5700 1 -- 172.21.15.43:0/1882398532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7901edaf0 msgr2=0x7ff7900c6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:30.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.806+0000 7ff7984d5700 1 --2- 172.21.15.43:0/1882398532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7901edaf0 0x7ff7900c6e60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff7800090b0 tx=0x7ff7800047c0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.806+0000 7ff7984d5700 1 -- 172.21.15.43:0/1882398532 shutdown_connections 2023-06-28T20:07:30.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.806+0000 7ff7984d5700 1 --2- 172.21.15.43:0/1882398532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7900c73a0 0x7ff7901589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.806+0000 7ff7984d5700 1 --2- 172.21.15.43:0/1882398532 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff77406ace0 0x7ff77406d190 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.806+0000 7ff7984d5700 1 --2- 172.21.15.43:0/1882398532 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff77c004120 0x7ff7900c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.806+0000 7ff7984d5700 1 --2- 172.21.15.43:0/1882398532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7901edaf0 0x7ff7900c6e60 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.806+0000 7ff7984d5700 1 -- 172.21.15.43:0/1882398532 >> 172.21.15.43:0/1882398532 conn(0x7ff79006c170 msgr2=0x7ff79015aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.806+0000 7ff7984d5700 1 -- 172.21.15.43:0/1882398532 shutdown_connections 2023-06-28T20:07:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.806+0000 7ff7984d5700 1 -- 172.21.15.43:0/1882398532 wait complete. 2023-06-28T20:07:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-06-28T20:07:30.821 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-06-28T20:07:29.993204+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218532,"kb_used_data":2916,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756188,"statfs":{"total":773094113280,"available":772870336512,"internally_reserved":0,"allocated":2985984,"data_stored":1817291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.631190"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381322+0000","last_change":"2023-06-28T20:07:21.392001+0000","last_active":"2023-06-28T20:07:22.381322+0000","last_peered":"2023-06-28T20:07:22.381322+0000","last_clean":"2023-06-28T20:07:22.381322+0000","last_became_active":"2023-06-28T20:07:21.391521+0000","last_became_peered":"2023-06-28T20:07:21.391521+0000","last_unstale":"2023-06-28T20:07:22.381322+0000","last_undegraded":"2023-06-28T20:07:22.381322+0000","last_fullsized":"2023-06-28T20:07:22.381322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:30:02.751518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787353+0000","last_change":"2023-06-28T20:07:21.381060+0000","last_active":"2023-06-28T20:07:23.787353+0000","last_peered":"2023-06-28T20:07:23.787353+0000","last_clean":"2023-06-28T20:07:23.787353+0000","last_became_active":"2023-06-28T20:07:21.380791+0000","last_became_peered":"2023-06-28T20:07:21.380791+0000","last_unstale":"2023-06-28T20:07:23.787353+0000","last_undegraded":"2023-06-28T20:07:23.787353+0000","last_fullsized":"2023-06-28T20:07:23.787353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:31:11.922720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200155+0000","last_change":"2023-06-28T20:07:21.375652+0000","last_active":"2023-06-28T20:07:23.200155+0000","last_peered":"2023-06-28T20:07:23.200155+0000","last_clean":"2023-06-28T20:07:23.200155+0000","last_became_active":"2023-06-28T20:07:21.375192+0000","last_became_peered":"2023-06-28T20:07:21.375192+0000","last_unstale":"2023-06-28T20:07:23.200155+0000","last_undegraded":"2023-06-28T20:07:23.200155+0000","last_fullsized":"2023-06-28T20:07:23.200155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:38:22.510874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382914+0000","last_change":"2023-06-28T20:07:21.376200+0000","last_active":"2023-06-28T20:07:22.382914+0000","last_peered":"2023-06-28T20:07:22.382914+0000","last_clean":"2023-06-28T20:07:22.382914+0000","last_became_active":"2023-06-28T20:07:21.375927+0000","last_became_peered":"2023-06-28T20:07:21.375927+0000","last_unstale":"2023-06-28T20:07:22.382914+0000","last_undegraded":"2023-06-28T20:07:22.382914+0000","last_fullsized":"2023-06-28T20:07:22.382914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:04:16.061920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786800+0000","last_change":"2023-06-28T20:07:21.394022+0000","last_active":"2023-06-28T20:07:23.786800+0000","last_peered":"2023-06-28T20:07:23.786800+0000","last_clean":"2023-06-28T20:07:23.786800+0000","last_became_active":"2023-06-28T20:07:21.393749+0000","last_became_peered":"2023-06-28T20:07:21.393749+0000","last_unstale":"2023-06-28T20:07:23.786800+0000","last_undegraded":"2023-06-28T20:07:23.786800+0000","last_fullsized":"2023-06-28T20:07:23.786800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:07:49.232842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.199922+0000","last_change":"2023-06-28T20:07:21.380285+0000","last_active":"2023-06-28T20:07:23.199922+0000","last_peered":"2023-06-28T20:07:23.199922+0000","last_clean":"2023-06-28T20:07:23.199922+0000","last_became_active":"2023-06-28T20:07:21.379978+0000","last_became_peered":"2023-06-28T20:07:21.379978+0000","last_unstale":"2023-06-28T20:07:23.199922+0000","last_undegraded":"2023-06-28T20:07:23.199922+0000","last_fullsized":"2023-06-28T20:07:23.199922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:50:32.581801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381458+0000","last_change":"2023-06-28T20:07:21.380983+0000","last_active":"2023-06-28T20:07:22.381458+0000","last_peered":"2023-06-28T20:07:22.381458+0000","last_clean":"2023-06-28T20:07:22.381458+0000","last_became_active":"2023-06-28T20:07:21.380706+0000","last_became_peered":"2023-06-28T20:07:21.380706+0000","last_unstale":"2023-06-28T20:07:22.381458+0000","last_undegraded":"2023-06-28T20:07:22.381458+0000","last_fullsized":"2023-06-28T20:07:22.381458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:43:51.244165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786970+0000","last_change":"2023-06-28T20:07:21.376404+0000","last_active":"2023-06-28T20:07:23.786970+0000","last_peered":"2023-06-28T20:07:23.786970+0000","last_clean":"2023-06-28T20:07:23.786970+0000","last_became_active":"2023-06-28T20:07:21.376136+0000","last_became_peered":"2023-06-28T20:07:21.376136+0000","last_unstale":"2023-06-28T20:07:23.786970+0000","last_undegraded":"2023-06-28T20:07:23.786970+0000","last_fullsized":"2023-06-28T20:07:23.786970+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:46:12.205864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787401+0000","last_change":"2023-06-28T20:07:21.394452+0000","last_active":"2023-06-28T20:07:23.787401+0000","last_peered":"2023-06-28T20:07:23.787401+0000","last_clean":"2023-06-28T20:07:23.787401+0000","last_became_active":"2023-06-28T20:07:21.393190+0000","last_became_peered":"2023-06-28T20:07:21.393190+0000","last_unstale":"2023-06-28T20:07:23.787401+0000","last_undegraded":"2023-06-28T20:07:23.787401+0000","last_fullsized":"2023-06-28T20:07:23.787401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:58:50.569412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787514+0000","last_change":"2023-06-28T20:07:21.384870+0000","last_active":"2023-06-28T20:07:23.787514+0000","last_peered":"2023-06-28T20:07:23.787514+0000","last_clean":"2023-06-28T20:07:23.787514+0000","last_became_active":"2023-06-28T20:07:21.384463+0000","last_became_peered":"2023-06-28T20:07:21.384463+0000","last_unstale":"2023-06-28T20:07:23.787514+0000","last_undegraded":"2023-06-28T20:07:23.787514+0000","last_fullsized":"2023-06-28T20:07:23.787514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:44:48.317033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200194+0000","last_change":"2023-06-28T20:07:21.381193+0000","last_active":"2023-06-28T20:07:23.200194+0000","last_peered":"2023-06-28T20:07:23.200194+0000","last_clean":"2023-06-28T20:07:23.200194+0000","last_became_active":"2023-06-28T20:07:21.380917+0000","last_became_peered":"2023-06-28T20:07:21.380917+0000","last_unstale":"2023-06-28T20:07:23.200194+0000","last_undegraded":"2023-06-28T20:07:23.200194+0000","last_fullsized":"2023-06-28T20:07:23.200194+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:14:14.828777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381377+0000","last_change":"2023-06-28T20:07:21.382341+0000","last_active":"2023-06-28T20:07:22.381377+0000","last_peered":"2023-06-28T20:07:22.381377+0000","last_clean":"2023-06-28T20:07:22.381377+0000","last_became_active":"2023-06-28T20:07:21.382032+0000","last_became_peered":"2023-06-28T20:07:21.382032+0000","last_unstale":"2023-06-28T20:07:22.381377+0000","last_undegraded":"2023-06-28T20:07:22.381377+0000","last_fullsized":"2023-06-28T20:07:22.381377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T08:00:53.732415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381426+0000","last_change":"2023-06-28T20:07:21.393006+0000","last_active":"2023-06-28T20:07:22.381426+0000","last_peered":"2023-06-28T20:07:22.381426+0000","last_clean":"2023-06-28T20:07:22.381426+0000","last_became_active":"2023-06-28T20:07:21.392710+0000","last_became_peered":"2023-06-28T20:07:21.392710+0000","last_unstale":"2023-06-28T20:07:22.381426+0000","last_undegraded":"2023-06-28T20:07:22.381426+0000","last_fullsized":"2023-06-28T20:07:22.381426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:01:21.282123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787052+0000","last_change":"2023-06-28T20:07:21.380173+0000","last_active":"2023-06-28T20:07:23.787052+0000","last_peered":"2023-06-28T20:07:23.787052+0000","last_clean":"2023-06-28T20:07:23.787052+0000","last_became_active":"2023-06-28T20:07:21.379821+0000","last_became_peered":"2023-06-28T20:07:21.379821+0000","last_unstale":"2023-06-28T20:07:23.787052+0000","last_undegraded":"2023-06-28T20:07:23.787052+0000","last_fullsized":"2023-06-28T20:07:23.787052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:14:35.290431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381515+0000","last_change":"2023-06-28T20:07:21.381063+0000","last_active":"2023-06-28T20:07:22.381515+0000","last_peered":"2023-06-28T20:07:22.381515+0000","last_clean":"2023-06-28T20:07:22.381515+0000","last_became_active":"2023-06-28T20:07:21.380785+0000","last_became_peered":"2023-06-28T20:07:21.380785+0000","last_unstale":"2023-06-28T20:07:22.381515+0000","last_undegraded":"2023-06-28T20:07:22.381515+0000","last_fullsized":"2023-06-28T20:07:22.381515+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:48:48.825002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.199680+0000","last_change":"2023-06-28T20:07:21.380326+0000","last_active":"2023-06-28T20:07:23.199680+0000","last_peered":"2023-06-28T20:07:23.199680+0000","last_clean":"2023-06-28T20:07:23.199680+0000","last_became_active":"2023-06-28T20:07:21.380043+0000","last_became_peered":"2023-06-28T20:07:21.380043+0000","last_unstale":"2023-06-28T20:07:23.199680+0000","last_undegraded":"2023-06-28T20:07:23.199680+0000","last_fullsized":"2023-06-28T20:07:23.199680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:27:06.937401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787442+0000","last_change":"2023-06-28T20:07:21.393453+0000","last_active":"2023-06-28T20:07:23.787442+0000","last_peered":"2023-06-28T20:07:23.787442+0000","last_clean":"2023-06-28T20:07:23.787442+0000","last_became_active":"2023-06-28T20:07:21.392856+0000","last_became_peered":"2023-06-28T20:07:21.392856+0000","last_unstale":"2023-06-28T20:07:23.787442+0000","last_undegraded":"2023-06-28T20:07:23.787442+0000","last_fullsized":"2023-06-28T20:07:23.787442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:44:48.317033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200392+0000","last_change":"2023-06-28T20:07:21.387823+0000","last_active":"2023-06-28T20:07:23.200392+0000","last_peered":"2023-06-28T20:07:23.200392+0000","last_clean":"2023-06-28T20:07:23.200392+0000","last_became_active":"2023-06-28T20:07:21.387528+0000","last_became_peered":"2023-06-28T20:07:21.387528+0000","last_unstale":"2023-06-28T20:07:23.200392+0000","last_undegraded":"2023-06-28T20:07:23.200392+0000","last_fullsized":"2023-06-28T20:07:23.200392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:05:11.078362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381385+0000","last_change":"2023-06-28T20:07:21.383174+0000","last_active":"2023-06-28T20:07:22.381385+0000","last_peered":"2023-06-28T20:07:22.381385+0000","last_clean":"2023-06-28T20:07:22.381385+0000","last_became_active":"2023-06-28T20:07:21.382865+0000","last_became_peered":"2023-06-28T20:07:21.382865+0000","last_unstale":"2023-06-28T20:07:22.381385+0000","last_undegraded":"2023-06-28T20:07:22.381385+0000","last_fullsized":"2023-06-28T20:07:22.381385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:19:03.691963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200053+0000","last_change":"2023-06-28T20:07:21.379462+0000","last_active":"2023-06-28T20:07:23.200053+0000","last_peered":"2023-06-28T20:07:23.200053+0000","last_clean":"2023-06-28T20:07:23.200053+0000","last_became_active":"2023-06-28T20:07:21.379190+0000","last_became_peered":"2023-06-28T20:07:21.379190+0000","last_unstale":"2023-06-28T20:07:23.200053+0000","last_undegraded":"2023-06-28T20:07:23.200053+0000","last_fullsized":"2023-06-28T20:07:23.200053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:48:48.825002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381211+0000","last_change":"2023-06-28T20:07:21.392576+0000","last_active":"2023-06-28T20:07:22.381211+0000","last_peered":"2023-06-28T20:07:22.381211+0000","last_clean":"2023-06-28T20:07:22.381211+0000","last_became_active":"2023-06-28T20:07:21.392374+0000","last_became_peered":"2023-06-28T20:07:21.392374+0000","last_unstale":"2023-06-28T20:07:22.381211+0000","last_undegraded":"2023-06-28T20:07:22.381211+0000","last_fullsized":"2023-06-28T20:07:22.381211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:56:31.654337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786384+0000","last_change":"2023-06-28T20:07:21.381495+0000","last_active":"2023-06-28T20:07:23.786384+0000","last_peered":"2023-06-28T20:07:23.786384+0000","last_clean":"2023-06-28T20:07:23.786384+0000","last_became_active":"2023-06-28T20:07:21.380960+0000","last_became_peered":"2023-06-28T20:07:21.380960+0000","last_unstale":"2023-06-28T20:07:23.786384+0000","last_undegraded":"2023-06-28T20:07:23.786384+0000","last_fullsized":"2023-06-28T20:07:23.786384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:16:04.779207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786669+0000","last_change":"2023-06-28T20:07:21.393139+0000","last_active":"2023-06-28T20:07:23.786669+0000","last_peered":"2023-06-28T20:07:23.786669+0000","last_clean":"2023-06-28T20:07:23.786669+0000","last_became_active":"2023-06-28T20:07:21.392866+0000","last_became_peered":"2023-06-28T20:07:21.392866+0000","last_unstale":"2023-06-28T20:07:23.786669+0000","last_undegraded":"2023-06-28T20:07:23.786669+0000","last_fullsized":"2023-06-28T20:07:23.786669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:55:38.748243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.380893+0000","last_change":"2023-06-28T20:07:21.386233+0000","last_active":"2023-06-28T20:07:22.380893+0000","last_peered":"2023-06-28T20:07:22.380893+0000","last_clean":"2023-06-28T20:07:22.380893+0000","last_became_active":"2023-06-28T20:07:21.385374+0000","last_became_peered":"2023-06-28T20:07:21.385374+0000","last_unstale":"2023-06-28T20:07:22.380893+0000","last_undegraded":"2023-06-28T20:07:22.380893+0000","last_fullsized":"2023-06-28T20:07:22.380893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:31:11.922720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.772271+0000","last_change":"2023-06-28T20:07:21.381465+0000","last_active":"2023-06-28T20:07:23.772271+0000","last_peered":"2023-06-28T20:07:23.772271+0000","last_clean":"2023-06-28T20:07:23.772271+0000","last_became_active":"2023-06-28T20:07:21.378500+0000","last_became_peered":"2023-06-28T20:07:21.378500+0000","last_unstale":"2023-06-28T20:07:23.772271+0000","last_undegraded":"2023-06-28T20:07:23.772271+0000","last_fullsized":"2023-06-28T20:07:23.772271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:40:59.613772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.772311+0000","last_change":"2023-06-28T20:07:21.384064+0000","last_active":"2023-06-28T20:07:23.772311+0000","last_peered":"2023-06-28T20:07:23.772311+0000","last_clean":"2023-06-28T20:07:23.772311+0000","last_became_active":"2023-06-28T20:07:21.383797+0000","last_became_peered":"2023-06-28T20:07:21.383797+0000","last_unstale":"2023-06-28T20:07:23.772311+0000","last_undegraded":"2023-06-28T20:07:23.772311+0000","last_fullsized":"2023-06-28T20:07:23.772311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:28:25.317531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787211+0000","last_change":"2023-06-28T20:07:21.382031+0000","last_active":"2023-06-28T20:07:23.787211+0000","last_peered":"2023-06-28T20:07:23.787211+0000","last_clean":"2023-06-28T20:07:23.787211+0000","last_became_active":"2023-06-28T20:07:21.381763+0000","last_became_peered":"2023-06-28T20:07:21.381763+0000","last_unstale":"2023-06-28T20:07:23.787211+0000","last_undegraded":"2023-06-28T20:07:23.787211+0000","last_fullsized":"2023-06-28T20:07:23.787211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:55:38.748243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200003+0000","last_change":"2023-06-28T20:07:20.382468+0000","last_active":"2023-06-28T20:07:23.200003+0000","last_peered":"2023-06-28T20:07:23.200003+0000","last_clean":"2023-06-28T20:07:23.200003+0000","last_became_active":"2023-06-28T20:07:20.382185+0000","last_became_peered":"2023-06-28T20:07:20.382185+0000","last_unstale":"2023-06-28T20:07:23.200003+0000","last_undegraded":"2023-06-28T20:07:23.200003+0000","last_fullsized":"2023-06-28T20:07:23.200003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:43:50.238683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787011+0000","last_change":"2023-06-28T20:07:21.386960+0000","last_active":"2023-06-28T20:07:23.787011+0000","last_peered":"2023-06-28T20:07:23.787011+0000","last_clean":"2023-06-28T20:07:23.787011+0000","last_became_active":"2023-06-28T20:07:21.386705+0000","last_became_peered":"2023-06-28T20:07:21.386705+0000","last_unstale":"2023-06-28T20:07:23.787011+0000","last_undegraded":"2023-06-28T20:07:23.787011+0000","last_fullsized":"2023-06-28T20:07:23.787011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:44:48.317033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.771954+0000","last_change":"2023-06-28T20:07:20.382275+0000","last_active":"2023-06-28T20:07:23.771954+0000","last_peered":"2023-06-28T20:07:23.771954+0000","last_clean":"2023-06-28T20:07:23.771954+0000","last_became_active":"2023-06-28T20:07:20.381923+0000","last_became_peered":"2023-06-28T20:07:20.381923+0000","last_unstale":"2023-06-28T20:07:23.771954+0000","last_undegraded":"2023-06-28T20:07:23.771954+0000","last_fullsized":"2023-06-28T20:07:23.771954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:14:34.284950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787292+0000","last_change":"2023-06-28T20:07:21.380199+0000","last_active":"2023-06-28T20:07:23.787292+0000","last_peered":"2023-06-28T20:07:23.787292+0000","last_clean":"2023-06-28T20:07:23.787292+0000","last_became_active":"2023-06-28T20:07:21.379925+0000","last_became_peered":"2023-06-28T20:07:21.379925+0000","last_unstale":"2023-06-28T20:07:23.787292+0000","last_undegraded":"2023-06-28T20:07:23.787292+0000","last_fullsized":"2023-06-28T20:07:23.787292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:28:25.317531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787110+0000","last_change":"2023-06-28T20:07:20.382465+0000","last_active":"2023-06-28T20:07:23.787110+0000","last_peered":"2023-06-28T20:07:23.787110+0000","last_clean":"2023-06-28T20:07:23.787110+0000","last_became_active":"2023-06-28T20:07:20.380273+0000","last_became_peered":"2023-06-28T20:07:20.380273+0000","last_unstale":"2023-06-28T20:07:23.787110+0000","last_undegraded":"2023-06-28T20:07:23.787110+0000","last_fullsized":"2023-06-28T20:07:23.787110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:27:05.931919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382428+0000","last_change":"2023-06-28T20:07:21.386881+0000","last_active":"2023-06-28T20:07:22.382428+0000","last_peered":"2023-06-28T20:07:22.382428+0000","last_clean":"2023-06-28T20:07:22.382428+0000","last_became_active":"2023-06-28T20:07:21.383395+0000","last_became_peered":"2023-06-28T20:07:21.383395+0000","last_unstale":"2023-06-28T20:07:22.382428+0000","last_undegraded":"2023-06-28T20:07:22.382428+0000","last_fullsized":"2023-06-28T20:07:22.382428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:34:26.678987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787530+0000","last_change":"2023-06-28T20:07:20.378567+0000","last_active":"2023-06-28T20:07:23.787530+0000","last_peered":"2023-06-28T20:07:23.787530+0000","last_clean":"2023-06-28T20:07:23.787530+0000","last_became_active":"2023-06-28T20:07:20.377624+0000","last_became_peered":"2023-06-28T20:07:20.377624+0000","last_unstale":"2023-06-28T20:07:23.787530+0000","last_undegraded":"2023-06-28T20:07:23.787530+0000","last_fullsized":"2023-06-28T20:07:23.787530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:21:31.960769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786981+0000","last_change":"2023-06-28T20:07:21.385250+0000","last_active":"2023-06-28T20:07:23.786981+0000","last_peered":"2023-06-28T20:07:23.786981+0000","last_clean":"2023-06-28T20:07:23.786981+0000","last_became_active":"2023-06-28T20:07:21.384763+0000","last_became_peered":"2023-06-28T20:07:21.384763+0000","last_unstale":"2023-06-28T20:07:23.786981+0000","last_undegraded":"2023-06-28T20:07:23.786981+0000","last_fullsized":"2023-06-28T20:07:23.786981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:58:50.569412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787334+0000","last_change":"2023-06-28T20:07:20.376535+0000","last_active":"2023-06-28T20:07:23.787334+0000","last_peered":"2023-06-28T20:07:23.787334+0000","last_clean":"2023-06-28T20:07:23.787334+0000","last_became_active":"2023-06-28T20:07:20.376182+0000","last_became_peered":"2023-06-28T20:07:20.376182+0000","last_unstale":"2023-06-28T20:07:23.787334+0000","last_undegraded":"2023-06-28T20:07:23.787334+0000","last_fullsized":"2023-06-28T20:07:23.787334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:02:41.673313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787356+0000","last_change":"2023-06-28T20:07:21.390529+0000","last_active":"2023-06-28T20:07:23.787356+0000","last_peered":"2023-06-28T20:07:23.787356+0000","last_clean":"2023-06-28T20:07:23.787356+0000","last_became_active":"2023-06-28T20:07:21.390249+0000","last_became_peered":"2023-06-28T20:07:21.390249+0000","last_unstale":"2023-06-28T20:07:23.787356+0000","last_undegraded":"2023-06-28T20:07:23.787356+0000","last_fullsized":"2023-06-28T20:07:23.787356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:43:51.244165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381357+0000","last_change":"2023-06-28T20:07:20.380921+0000","last_active":"2023-06-28T20:07:22.381357+0000","last_peered":"2023-06-28T20:07:22.381357+0000","last_clean":"2023-06-28T20:07:22.381357+0000","last_became_active":"2023-06-28T20:07:20.380632+0000","last_became_peered":"2023-06-28T20:07:20.380632+0000","last_unstale":"2023-06-28T20:07:22.381357+0000","last_undegraded":"2023-06-28T20:07:22.381357+0000","last_fullsized":"2023-06-28T20:07:22.381357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:03:07.390037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.772038+0000","last_change":"2023-06-28T20:07:21.379317+0000","last_active":"2023-06-28T20:07:23.772038+0000","last_peered":"2023-06-28T20:07:23.772038+0000","last_clean":"2023-06-28T20:07:23.772038+0000","last_became_active":"2023-06-28T20:07:21.379040+0000","last_became_peered":"2023-06-28T20:07:21.379040+0000","last_unstale":"2023-06-28T20:07:23.772038+0000","last_undegraded":"2023-06-28T20:07:23.772038+0000","last_fullsized":"2023-06-28T20:07:23.772038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:04:16.061920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200223+0000","last_change":"2023-06-28T20:07:20.376485+0000","last_active":"2023-06-28T20:07:23.200223+0000","last_peered":"2023-06-28T20:07:23.200223+0000","last_clean":"2023-06-28T20:07:23.200223+0000","last_became_active":"2023-06-28T20:07:20.376240+0000","last_became_peered":"2023-06-28T20:07:20.376240+0000","last_unstale":"2023-06-28T20:07:23.200223+0000","last_undegraded":"2023-06-28T20:07:23.200223+0000","last_fullsized":"2023-06-28T20:07:23.200223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:06:43.811286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200223+0000","last_change":"2023-06-28T20:07:21.379150+0000","last_active":"2023-06-28T20:07:23.200223+0000","last_peered":"2023-06-28T20:07:23.200223+0000","last_clean":"2023-06-28T20:07:23.200223+0000","last_became_active":"2023-06-28T20:07:21.378850+0000","last_became_peered":"2023-06-28T20:07:21.378850+0000","last_unstale":"2023-06-28T20:07:23.200223+0000","last_undegraded":"2023-06-28T20:07:23.200223+0000","last_fullsized":"2023-06-28T20:07:23.200223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:29:17.800360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381032+0000","last_change":"2023-06-28T20:07:20.378538+0000","last_active":"2023-06-28T20:07:22.381032+0000","last_peered":"2023-06-28T20:07:22.381032+0000","last_clean":"2023-06-28T20:07:22.381032+0000","last_became_active":"2023-06-28T20:07:20.378180+0000","last_became_peered":"2023-06-28T20:07:20.378180+0000","last_unstale":"2023-06-28T20:07:22.381032+0000","last_undegraded":"2023-06-28T20:07:22.381032+0000","last_fullsized":"2023-06-28T20:07:22.381032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:58:49.563930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786664+0000","last_change":"2023-06-28T20:07:21.389643+0000","last_active":"2023-06-28T20:07:23.786664+0000","last_peered":"2023-06-28T20:07:23.786664+0000","last_clean":"2023-06-28T20:07:23.786664+0000","last_became_active":"2023-06-28T20:07:21.389163+0000","last_became_peered":"2023-06-28T20:07:21.389163+0000","last_unstale":"2023-06-28T20:07:23.786664+0000","last_undegraded":"2023-06-28T20:07:23.786664+0000","last_fullsized":"2023-06-28T20:07:23.786664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:07:49.232842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786561+0000","last_change":"2023-06-28T20:07:20.371972+0000","last_active":"2023-06-28T20:07:23.786561+0000","last_peered":"2023-06-28T20:07:23.786561+0000","last_clean":"2023-06-28T20:07:23.786561+0000","last_became_active":"2023-06-28T20:07:20.370619+0000","last_became_peered":"2023-06-28T20:07:20.370619+0000","last_unstale":"2023-06-28T20:07:23.786561+0000","last_undegraded":"2023-06-28T20:07:23.786561+0000","last_fullsized":"2023-06-28T20:07:23.786561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:34:25.673506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787272+0000","last_change":"2023-06-28T20:07:21.382142+0000","last_active":"2023-06-28T20:07:23.787272+0000","last_peered":"2023-06-28T20:07:23.787272+0000","last_clean":"2023-06-28T20:07:23.787272+0000","last_became_active":"2023-06-28T20:07:21.381848+0000","last_became_peered":"2023-06-28T20:07:21.381848+0000","last_unstale":"2023-06-28T20:07:23.787272+0000","last_undegraded":"2023-06-28T20:07:23.787272+0000","last_fullsized":"2023-06-28T20:07:23.787272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:40:59.613772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381309+0000","last_change":"2023-06-28T20:07:20.376749+0000","last_active":"2023-06-28T20:07:22.381309+0000","last_peered":"2023-06-28T20:07:22.381309+0000","last_clean":"2023-06-28T20:07:22.381309+0000","last_became_active":"2023-06-28T20:07:20.376465+0000","last_became_peered":"2023-06-28T20:07:20.376465+0000","last_unstale":"2023-06-28T20:07:22.381309+0000","last_undegraded":"2023-06-28T20:07:22.381309+0000","last_fullsized":"2023-06-28T20:07:22.381309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:20:39.309655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787027+0000","last_change":"2023-06-28T20:07:21.393127+0000","last_active":"2023-06-28T20:07:23.787027+0000","last_peered":"2023-06-28T20:07:23.787027+0000","last_clean":"2023-06-28T20:07:23.787027+0000","last_became_active":"2023-06-28T20:07:21.392832+0000","last_became_peered":"2023-06-28T20:07:21.392832+0000","last_unstale":"2023-06-28T20:07:23.787027+0000","last_undegraded":"2023-06-28T20:07:23.787027+0000","last_fullsized":"2023-06-28T20:07:23.787027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:40:59.613772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381838+0000","last_change":"2023-06-28T20:07:20.380053+0000","last_active":"2023-06-28T20:07:22.381838+0000","last_peered":"2023-06-28T20:07:22.381838+0000","last_clean":"2023-06-28T20:07:22.381838+0000","last_became_active":"2023-06-28T20:07:20.379752+0000","last_became_peered":"2023-06-28T20:07:20.379752+0000","last_unstale":"2023-06-28T20:07:22.381838+0000","last_undegraded":"2023-06-28T20:07:22.381838+0000","last_fullsized":"2023-06-28T20:07:22.381838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:58:14.638684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787470+0000","last_change":"2023-06-28T20:07:21.390679+0000","last_active":"2023-06-28T20:07:23.787470+0000","last_peered":"2023-06-28T20:07:23.787470+0000","last_clean":"2023-06-28T20:07:23.787470+0000","last_became_active":"2023-06-28T20:07:21.390332+0000","last_became_peered":"2023-06-28T20:07:21.390332+0000","last_unstale":"2023-06-28T20:07:23.787470+0000","last_undegraded":"2023-06-28T20:07:23.787470+0000","last_fullsized":"2023-06-28T20:07:23.787470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:48:48.825002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381372+0000","last_change":"2023-06-28T20:07:20.379001+0000","last_active":"2023-06-28T20:07:22.381372+0000","last_peered":"2023-06-28T20:07:22.381372+0000","last_clean":"2023-06-28T20:07:22.381372+0000","last_became_active":"2023-06-28T20:07:20.378692+0000","last_became_peered":"2023-06-28T20:07:20.378692+0000","last_unstale":"2023-06-28T20:07:22.381372+0000","last_undegraded":"2023-06-28T20:07:22.381372+0000","last_fullsized":"2023-06-28T20:07:22.381372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:50:31.576319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786912+0000","last_change":"2023-06-28T20:07:21.377599+0000","last_active":"2023-06-28T20:07:23.786912+0000","last_peered":"2023-06-28T20:07:23.786912+0000","last_clean":"2023-06-28T20:07:23.786912+0000","last_became_active":"2023-06-28T20:07:21.377323+0000","last_became_peered":"2023-06-28T20:07:21.377323+0000","last_unstale":"2023-06-28T20:07:23.786912+0000","last_undegraded":"2023-06-28T20:07:23.786912+0000","last_fullsized":"2023-06-28T20:07:23.786912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:34:26.678987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381358+0000","last_change":"2023-06-28T20:07:20.379564+0000","last_active":"2023-06-28T20:07:22.381358+0000","last_peered":"2023-06-28T20:07:22.381358+0000","last_clean":"2023-06-28T20:07:22.381358+0000","last_became_active":"2023-06-28T20:07:20.379179+0000","last_became_peered":"2023-06-28T20:07:20.379179+0000","last_unstale":"2023-06-28T20:07:22.381358+0000","last_undegraded":"2023-06-28T20:07:22.381358+0000","last_fullsized":"2023-06-28T20:07:22.381358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:44:14.718714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.199692+0000","last_change":"2023-06-28T20:07:21.375868+0000","last_active":"2023-06-28T20:07:23.199692+0000","last_peered":"2023-06-28T20:07:23.199692+0000","last_clean":"2023-06-28T20:07:23.199692+0000","last_became_active":"2023-06-28T20:07:21.375273+0000","last_became_peered":"2023-06-28T20:07:21.375273+0000","last_unstale":"2023-06-28T20:07:23.199692+0000","last_undegraded":"2023-06-28T20:07:23.199692+0000","last_fullsized":"2023-06-28T20:07:23.199692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:44:48.317033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381183+0000","last_change":"2023-06-28T20:07:20.377080+0000","last_active":"2023-06-28T20:07:22.381183+0000","last_peered":"2023-06-28T20:07:22.381183+0000","last_clean":"2023-06-28T20:07:22.381183+0000","last_became_active":"2023-06-28T20:07:20.376266+0000","last_became_peered":"2023-06-28T20:07:20.376266+0000","last_unstale":"2023-06-28T20:07:22.381183+0000","last_undegraded":"2023-06-28T20:07:22.381183+0000","last_fullsized":"2023-06-28T20:07:22.381183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:21:31.960769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382189+0000","last_change":"2023-06-28T20:07:21.382463+0000","last_active":"2023-06-28T20:07:22.382189+0000","last_peered":"2023-06-28T20:07:22.382189+0000","last_clean":"2023-06-28T20:07:22.382189+0000","last_became_active":"2023-06-28T20:07:21.382163+0000","last_became_peered":"2023-06-28T20:07:21.382163+0000","last_unstale":"2023-06-28T20:07:22.382189+0000","last_undegraded":"2023-06-28T20:07:22.382189+0000","last_fullsized":"2023-06-28T20:07:22.382189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:44:41.666645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787191+0000","last_change":"2023-06-28T20:07:20.380798+0000","last_active":"2023-06-28T20:07:23.787191+0000","last_peered":"2023-06-28T20:07:23.787191+0000","last_clean":"2023-06-28T20:07:23.787191+0000","last_became_active":"2023-06-28T20:07:20.380380+0000","last_became_peered":"2023-06-28T20:07:20.380380+0000","last_unstale":"2023-06-28T20:07:23.787191+0000","last_undegraded":"2023-06-28T20:07:23.787191+0000","last_fullsized":"2023-06-28T20:07:23.787191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:07:48.227360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786571+0000","last_change":"2023-06-28T20:07:21.382097+0000","last_active":"2023-06-28T20:07:23.786571+0000","last_peered":"2023-06-28T20:07:23.786571+0000","last_clean":"2023-06-28T20:07:23.786571+0000","last_became_active":"2023-06-28T20:07:21.381678+0000","last_became_peered":"2023-06-28T20:07:21.381678+0000","last_unstale":"2023-06-28T20:07:23.786571+0000","last_undegraded":"2023-06-28T20:07:23.786571+0000","last_fullsized":"2023-06-28T20:07:23.786571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:14:35.290431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787297+0000","last_change":"2023-06-28T20:07:20.380519+0000","last_active":"2023-06-28T20:07:23.787297+0000","last_peered":"2023-06-28T20:07:23.787297+0000","last_clean":"2023-06-28T20:07:23.787297+0000","last_became_active":"2023-06-28T20:07:20.380219+0000","last_became_peered":"2023-06-28T20:07:20.380219+0000","last_unstale":"2023-06-28T20:07:23.787297+0000","last_undegraded":"2023-06-28T20:07:23.787297+0000","last_fullsized":"2023-06-28T20:07:23.787297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:50:31.576319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200096+0000","last_change":"2023-06-28T20:07:21.391559+0000","last_active":"2023-06-28T20:07:23.200096+0000","last_peered":"2023-06-28T20:07:23.200096+0000","last_clean":"2023-06-28T20:07:23.200096+0000","last_became_active":"2023-06-28T20:07:21.391214+0000","last_became_peered":"2023-06-28T20:07:21.391214+0000","last_unstale":"2023-06-28T20:07:23.200096+0000","last_undegraded":"2023-06-28T20:07:23.200096+0000","last_fullsized":"2023-06-28T20:07:23.200096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:07:11.795544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382144+0000","last_change":"2023-06-28T20:07:20.378412+0000","last_active":"2023-06-28T20:07:22.382144+0000","last_peered":"2023-06-28T20:07:22.382144+0000","last_clean":"2023-06-28T20:07:22.382144+0000","last_became_active":"2023-06-28T20:07:20.378102+0000","last_became_peered":"2023-06-28T20:07:20.378102+0000","last_unstale":"2023-06-28T20:07:22.382144+0000","last_undegraded":"2023-06-28T20:07:22.382144+0000","last_fullsized":"2023-06-28T20:07:22.382144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:05:45.610772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200420+0000","last_change":"2023-06-28T20:07:21.387960+0000","last_active":"2023-06-28T20:07:23.200420+0000","last_peered":"2023-06-28T20:07:23.200420+0000","last_clean":"2023-06-28T20:07:23.200420+0000","last_became_active":"2023-06-28T20:07:21.387590+0000","last_became_peered":"2023-06-28T20:07:21.387590+0000","last_unstale":"2023-06-28T20:07:23.200420+0000","last_undegraded":"2023-06-28T20:07:23.200420+0000","last_fullsized":"2023-06-28T20:07:23.200420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:06:39.112346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.199929+0000","last_change":"2023-06-28T20:07:20.378036+0000","last_active":"2023-06-28T20:07:23.199929+0000","last_peered":"2023-06-28T20:07:23.199929+0000","last_clean":"2023-06-28T20:07:23.199929+0000","last_became_active":"2023-06-28T20:07:20.377739+0000","last_became_peered":"2023-06-28T20:07:20.377739+0000","last_unstale":"2023-06-28T20:07:23.199929+0000","last_undegraded":"2023-06-28T20:07:23.199929+0000","last_fullsized":"2023-06-28T20:07:23.199929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:44:14.718714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200461+0000","last_change":"2023-06-28T20:07:21.379387+0000","last_active":"2023-06-28T20:07:23.200461+0000","last_peered":"2023-06-28T20:07:23.200461+0000","last_clean":"2023-06-28T20:07:23.200461+0000","last_became_active":"2023-06-28T20:07:21.379090+0000","last_became_peered":"2023-06-28T20:07:21.379090+0000","last_unstale":"2023-06-28T20:07:23.200461+0000","last_undegraded":"2023-06-28T20:07:23.200461+0000","last_fullsized":"2023-06-28T20:07:23.200461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:58:57.345862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786825+0000","last_change":"2023-06-28T20:07:20.382672+0000","last_active":"2023-06-28T20:07:23.786825+0000","last_peered":"2023-06-28T20:07:23.786825+0000","last_clean":"2023-06-28T20:07:23.786825+0000","last_became_active":"2023-06-28T20:07:20.381085+0000","last_became_peered":"2023-06-28T20:07:20.381085+0000","last_unstale":"2023-06-28T20:07:23.786825+0000","last_undegraded":"2023-06-28T20:07:23.786825+0000","last_fullsized":"2023-06-28T20:07:23.786825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:28:24.312050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.772421+0000","last_change":"2023-06-28T20:07:21.386618+0000","last_active":"2023-06-28T20:07:23.772421+0000","last_peered":"2023-06-28T20:07:23.772421+0000","last_clean":"2023-06-28T20:07:23.772421+0000","last_became_active":"2023-06-28T20:07:21.386002+0000","last_became_peered":"2023-06-28T20:07:21.386002+0000","last_unstale":"2023-06-28T20:07:23.772421+0000","last_undegraded":"2023-06-28T20:07:23.772421+0000","last_fullsized":"2023-06-28T20:07:23.772421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:44:48.317033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787225+0000","last_change":"2023-06-28T20:07:20.379096+0000","last_active":"2023-06-28T20:07:23.787225+0000","last_peered":"2023-06-28T20:07:23.787225+0000","last_clean":"2023-06-28T20:07:23.787225+0000","last_became_active":"2023-06-28T20:07:20.378681+0000","last_became_peered":"2023-06-28T20:07:20.378681+0000","last_unstale":"2023-06-28T20:07:23.787225+0000","last_undegraded":"2023-06-28T20:07:23.787225+0000","last_fullsized":"2023-06-28T20:07:23.787225+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:03:07.390037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200311+0000","last_change":"2023-06-28T20:07:15.815425+0000","last_active":"2023-06-28T20:07:23.200311+0000","last_peered":"2023-06-28T20:07:23.200311+0000","last_clean":"2023-06-28T20:07:23.200311+0000","last_became_active":"2023-06-28T20:07:15.815163+0000","last_became_peered":"2023-06-28T20:07:15.815163+0000","last_unstale":"2023-06-28T20:07:23.200311+0000","last_undegraded":"2023-06-28T20:07:23.200311+0000","last_fullsized":"2023-06-28T20:07:23.200311+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:50:06.853689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786694+0000","last_change":"2023-06-28T20:07:21.385061+0000","last_active":"2023-06-28T20:07:23.786694+0000","last_peered":"2023-06-28T20:07:23.786694+0000","last_clean":"2023-06-28T20:07:23.786694+0000","last_became_active":"2023-06-28T20:07:21.384404+0000","last_became_peered":"2023-06-28T20:07:21.384404+0000","last_unstale":"2023-06-28T20:07:23.786694+0000","last_undegraded":"2023-06-28T20:07:23.786694+0000","last_fullsized":"2023-06-28T20:07:23.786694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:40:59.613772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786799+0000","last_change":"2023-06-28T20:07:15.303675+0000","last_active":"2023-06-28T20:07:23.786799+0000","last_peered":"2023-06-28T20:07:23.786799+0000","last_clean":"2023-06-28T20:07:23.786799+0000","last_became_active":"2023-06-28T20:07:15.303350+0000","last_became_peered":"2023-06-28T20:07:15.303350+0000","last_unstale":"2023-06-28T20:07:23.786799+0000","last_undegraded":"2023-06-28T20:07:23.786799+0000","last_fullsized":"2023-06-28T20:07:23.786799+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:02:36.611170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"21'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787567+0000","last_change":"2023-06-28T20:07:20.381457+0000","last_active":"2023-06-28T20:07:23.787567+0000","last_peered":"2023-06-28T20:07:23.787567+0000","last_clean":"2023-06-28T20:07:23.787567+0000","last_became_active":"2023-06-28T20:07:20.381235+0000","last_became_peered":"2023-06-28T20:07:20.381235+0000","last_unstale":"2023-06-28T20:07:23.787567+0000","last_undegraded":"2023-06-28T20:07:23.787567+0000","last_fullsized":"2023-06-28T20:07:23.787567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:31:10.917238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.199885+0000","last_change":"2023-06-28T20:07:21.383647+0000","last_active":"2023-06-28T20:07:23.199885+0000","last_peered":"2023-06-28T20:07:23.199885+0000","last_clean":"2023-06-28T20:07:23.199885+0000","last_became_active":"2023-06-28T20:07:21.383392+0000","last_became_peered":"2023-06-28T20:07:21.383392+0000","last_unstale":"2023-06-28T20:07:23.199885+0000","last_undegraded":"2023-06-28T20:07:23.199885+0000","last_fullsized":"2023-06-28T20:07:23.199885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:03:08.395518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.771713+0000","last_change":"2023-06-28T20:07:15.303565+0000","last_active":"2023-06-28T20:07:23.771713+0000","last_peered":"2023-06-28T20:07:23.771713+0000","last_clean":"2023-06-28T20:07:23.771713+0000","last_became_active":"2023-06-28T20:07:15.303200+0000","last_became_peered":"2023-06-28T20:07:15.303200+0000","last_unstale":"2023-06-28T20:07:23.771713+0000","last_undegraded":"2023-06-28T20:07:23.771713+0000","last_fullsized":"2023-06-28T20:07:23.771713+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:46:06.138239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":169,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381052+0000","last_change":"2023-06-28T20:07:15.304921+0000","last_active":"2023-06-28T20:07:22.381052+0000","last_peered":"2023-06-28T20:07:22.381052+0000","last_clean":"2023-06-28T20:07:22.381052+0000","last_became_active":"2023-06-28T20:07:15.303959+0000","last_became_peered":"2023-06-28T20:07:15.303959+0000","last_unstale":"2023-06-28T20:07:22.381052+0000","last_undegraded":"2023-06-28T20:07:22.381052+0000","last_fullsized":"2023-06-28T20:07:22.381052+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:44:42.249408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786530+0000","last_change":"2023-06-28T20:07:20.379375+0000","last_active":"2023-06-28T20:07:23.786530+0000","last_peered":"2023-06-28T20:07:23.786530+0000","last_clean":"2023-06-28T20:07:23.786530+0000","last_became_active":"2023-06-28T20:07:20.378992+0000","last_became_peered":"2023-06-28T20:07:20.378992+0000","last_unstale":"2023-06-28T20:07:23.786530+0000","last_undegraded":"2023-06-28T20:07:23.786530+0000","last_fullsized":"2023-06-28T20:07:23.786530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:46:11.200383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787157+0000","last_change":"2023-06-28T20:07:21.383270+0000","last_active":"2023-06-28T20:07:23.787157+0000","last_peered":"2023-06-28T20:07:23.787157+0000","last_clean":"2023-06-28T20:07:23.787157+0000","last_became_active":"2023-06-28T20:07:21.382983+0000","last_became_peered":"2023-06-28T20:07:21.382983+0000","last_unstale":"2023-06-28T20:07:23.787157+0000","last_undegraded":"2023-06-28T20:07:23.787157+0000","last_fullsized":"2023-06-28T20:07:23.787157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:02:42.678795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.380861+0000","last_change":"2023-06-28T20:07:15.899048+0000","last_active":"2023-06-28T20:07:22.380861+0000","last_peered":"2023-06-28T20:07:22.380861+0000","last_clean":"2023-06-28T20:07:22.380861+0000","last_became_active":"2023-06-28T20:07:15.898766+0000","last_became_peered":"2023-06-28T20:07:15.898766+0000","last_unstale":"2023-06-28T20:07:22.380861+0000","last_undegraded":"2023-06-28T20:07:22.380861+0000","last_fullsized":"2023-06-28T20:07:22.380861+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:27:00.869776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.771898+0000","last_change":"2023-06-28T20:07:20.369864+0000","last_active":"2023-06-28T20:07:23.771898+0000","last_peered":"2023-06-28T20:07:23.771898+0000","last_clean":"2023-06-28T20:07:23.771898+0000","last_became_active":"2023-06-28T20:07:20.369542+0000","last_became_peered":"2023-06-28T20:07:20.369542+0000","last_unstale":"2023-06-28T20:07:23.771898+0000","last_undegraded":"2023-06-28T20:07:23.771898+0000","last_fullsized":"2023-06-28T20:07:23.771898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:34:25.673506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.772302+0000","last_change":"2023-06-28T20:07:21.383019+0000","last_active":"2023-06-28T20:07:23.772302+0000","last_peered":"2023-06-28T20:07:23.772302+0000","last_clean":"2023-06-28T20:07:23.772302+0000","last_became_active":"2023-06-28T20:07:21.382756+0000","last_became_peered":"2023-06-28T20:07:21.382756+0000","last_unstale":"2023-06-28T20:07:23.772302+0000","last_undegraded":"2023-06-28T20:07:23.772302+0000","last_fullsized":"2023-06-28T20:07:23.772302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:02:42.678795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.380789+0000","last_change":"2023-06-28T20:07:15.899229+0000","last_active":"2023-06-28T20:07:22.380789+0000","last_peered":"2023-06-28T20:07:22.380789+0000","last_clean":"2023-06-28T20:07:22.380789+0000","last_became_active":"2023-06-28T20:07:15.898708+0000","last_became_peered":"2023-06-28T20:07:15.898708+0000","last_unstale":"2023-06-28T20:07:22.380789+0000","last_undegraded":"2023-06-28T20:07:22.380789+0000","last_fullsized":"2023-06-28T20:07:22.380789+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:28:19.249906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787350+0000","last_change":"2023-06-28T20:07:20.381183+0000","last_active":"2023-06-28T20:07:23.787350+0000","last_peered":"2023-06-28T20:07:23.787350+0000","last_clean":"2023-06-28T20:07:23.787350+0000","last_became_active":"2023-06-28T20:07:20.380859+0000","last_became_peered":"2023-06-28T20:07:20.380859+0000","last_unstale":"2023-06-28T20:07:23.787350+0000","last_undegraded":"2023-06-28T20:07:23.787350+0000","last_fullsized":"2023-06-28T20:07:23.787350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:44:14.718714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382351+0000","last_change":"2023-06-28T20:07:21.382900+0000","last_active":"2023-06-28T20:07:22.382351+0000","last_peered":"2023-06-28T20:07:22.382351+0000","last_clean":"2023-06-28T20:07:22.382351+0000","last_became_active":"2023-06-28T20:07:21.382384+0000","last_became_peered":"2023-06-28T20:07:21.382384+0000","last_unstale":"2023-06-28T20:07:22.382351+0000","last_undegraded":"2023-06-28T20:07:22.382351+0000","last_fullsized":"2023-06-28T20:07:22.382351+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:16:04.779207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786990+0000","last_change":"2023-06-28T20:07:15.898966+0000","last_active":"2023-06-28T20:07:23.786990+0000","last_peered":"2023-06-28T20:07:23.786990+0000","last_clean":"2023-06-28T20:07:23.786990+0000","last_became_active":"2023-06-28T20:07:15.898704+0000","last_became_peered":"2023-06-28T20:07:15.898704+0000","last_unstale":"2023-06-28T20:07:23.786990+0000","last_undegraded":"2023-06-28T20:07:23.786990+0000","last_fullsized":"2023-06-28T20:07:23.786990+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:15:58.711582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.199867+0000","last_change":"2023-06-28T20:07:20.378848+0000","last_active":"2023-06-28T20:07:23.199867+0000","last_peered":"2023-06-28T20:07:23.199867+0000","last_clean":"2023-06-28T20:07:23.199867+0000","last_became_active":"2023-06-28T20:07:20.378438+0000","last_became_peered":"2023-06-28T20:07:20.378438+0000","last_unstale":"2023-06-28T20:07:23.199867+0000","last_undegraded":"2023-06-28T20:07:23.199867+0000","last_fullsized":"2023-06-28T20:07:23.199867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:20:39.309655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.772343+0000","last_change":"2023-06-28T20:07:21.384171+0000","last_active":"2023-06-28T20:07:23.772343+0000","last_peered":"2023-06-28T20:07:23.772343+0000","last_clean":"2023-06-28T20:07:23.772343+0000","last_became_active":"2023-06-28T20:07:21.383867+0000","last_became_peered":"2023-06-28T20:07:21.383867+0000","last_unstale":"2023-06-28T20:07:23.772343+0000","last_undegraded":"2023-06-28T20:07:23.772343+0000","last_fullsized":"2023-06-28T20:07:23.772343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:58:50.569412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382412+0000","last_change":"2023-06-28T20:07:15.899207+0000","last_active":"2023-06-28T20:07:22.382412+0000","last_peered":"2023-06-28T20:07:22.382412+0000","last_clean":"2023-06-28T20:07:22.382412+0000","last_became_active":"2023-06-28T20:07:15.898908+0000","last_became_peered":"2023-06-28T20:07:15.898908+0000","last_unstale":"2023-06-28T20:07:22.382412+0000","last_undegraded":"2023-06-28T20:07:22.382412+0000","last_fullsized":"2023-06-28T20:07:22.382412+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:14:29.222806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.772318+0000","last_change":"2023-06-28T20:07:20.377530+0000","last_active":"2023-06-28T20:07:23.772318+0000","last_peered":"2023-06-28T20:07:23.772318+0000","last_clean":"2023-06-28T20:07:23.772318+0000","last_became_active":"2023-06-28T20:07:20.377243+0000","last_became_peered":"2023-06-28T20:07:20.377243+0000","last_unstale":"2023-06-28T20:07:23.772318+0000","last_undegraded":"2023-06-28T20:07:23.772318+0000","last_fullsized":"2023-06-28T20:07:23.772318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:31:10.917238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786957+0000","last_change":"2023-06-28T20:07:21.383886+0000","last_active":"2023-06-28T20:07:23.786957+0000","last_peered":"2023-06-28T20:07:23.786957+0000","last_clean":"2023-06-28T20:07:23.786957+0000","last_became_active":"2023-06-28T20:07:21.383572+0000","last_became_peered":"2023-06-28T20:07:21.383572+0000","last_unstale":"2023-06-28T20:07:23.786957+0000","last_undegraded":"2023-06-28T20:07:23.786957+0000","last_fullsized":"2023-06-28T20:07:23.786957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:55:38.748243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382342+0000","last_change":"2023-06-28T20:07:15.303873+0000","last_active":"2023-06-28T20:07:22.382342+0000","last_peered":"2023-06-28T20:07:22.382342+0000","last_clean":"2023-06-28T20:07:22.382342+0000","last_became_active":"2023-06-28T20:07:15.303457+0000","last_became_peered":"2023-06-28T20:07:15.303457+0000","last_unstale":"2023-06-28T20:07:22.382342+0000","last_undegraded":"2023-06-28T20:07:22.382342+0000","last_fullsized":"2023-06-28T20:07:22.382342+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:46:06.138239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787461+0000","last_change":"2023-06-28T20:07:20.381529+0000","last_active":"2023-06-28T20:07:23.787461+0000","last_peered":"2023-06-28T20:07:23.787461+0000","last_clean":"2023-06-28T20:07:23.787461+0000","last_became_active":"2023-06-28T20:07:20.381191+0000","last_became_peered":"2023-06-28T20:07:20.381191+0000","last_unstale":"2023-06-28T20:07:23.787461+0000","last_undegraded":"2023-06-28T20:07:23.787461+0000","last_fullsized":"2023-06-28T20:07:23.787461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:58:49.563930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.772112+0000","last_change":"2023-06-28T20:07:21.377519+0000","last_active":"2023-06-28T20:07:23.772112+0000","last_peered":"2023-06-28T20:07:23.772112+0000","last_clean":"2023-06-28T20:07:23.772112+0000","last_became_active":"2023-06-28T20:07:21.377225+0000","last_became_peered":"2023-06-28T20:07:21.377225+0000","last_unstale":"2023-06-28T20:07:23.772112+0000","last_undegraded":"2023-06-28T20:07:23.772112+0000","last_fullsized":"2023-06-28T20:07:23.772112+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:07:49.232842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786710+0000","last_change":"2023-06-28T20:07:20.382678+0000","last_active":"2023-06-28T20:07:23.786710+0000","last_peered":"2023-06-28T20:07:23.786710+0000","last_clean":"2023-06-28T20:07:23.786710+0000","last_became_active":"2023-06-28T20:07:20.382361+0000","last_became_peered":"2023-06-28T20:07:20.382361+0000","last_unstale":"2023-06-28T20:07:23.786710+0000","last_undegraded":"2023-06-28T20:07:23.786710+0000","last_fullsized":"2023-06-28T20:07:23.786710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:04:15.056439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.381381+0000","last_change":"2023-06-28T20:07:21.397232+0000","last_active":"2023-06-28T20:07:22.381381+0000","last_peered":"2023-06-28T20:07:22.381381+0000","last_clean":"2023-06-28T20:07:22.381381+0000","last_became_active":"2023-06-28T20:07:21.396862+0000","last_became_peered":"2023-06-28T20:07:21.396862+0000","last_unstale":"2023-06-28T20:07:22.381381+0000","last_undegraded":"2023-06-28T20:07:22.381381+0000","last_fullsized":"2023-06-28T20:07:22.381381+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:48:55.783926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787099+0000","last_change":"2023-06-28T20:07:20.380628+0000","last_active":"2023-06-28T20:07:23.787099+0000","last_peered":"2023-06-28T20:07:23.787099+0000","last_clean":"2023-06-28T20:07:23.787099+0000","last_became_active":"2023-06-28T20:07:20.380033+0000","last_became_peered":"2023-06-28T20:07:20.380033+0000","last_unstale":"2023-06-28T20:07:23.787099+0000","last_undegraded":"2023-06-28T20:07:23.787099+0000","last_fullsized":"2023-06-28T20:07:23.787099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:04:15.056439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.199812+0000","last_change":"2023-06-28T20:07:21.374666+0000","last_active":"2023-06-28T20:07:23.199812+0000","last_peered":"2023-06-28T20:07:23.199812+0000","last_clean":"2023-06-28T20:07:23.199812+0000","last_became_active":"2023-06-28T20:07:21.374330+0000","last_became_peered":"2023-06-28T20:07:21.374330+0000","last_unstale":"2023-06-28T20:07:23.199812+0000","last_undegraded":"2023-06-28T20:07:23.199812+0000","last_fullsized":"2023-06-28T20:07:23.199812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:21:32.966251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787460+0000","last_change":"2023-06-28T20:07:20.378725+0000","last_active":"2023-06-28T20:07:23.787460+0000","last_peered":"2023-06-28T20:07:23.787460+0000","last_clean":"2023-06-28T20:07:23.787460+0000","last_became_active":"2023-06-28T20:07:20.378445+0000","last_became_peered":"2023-06-28T20:07:20.378445+0000","last_unstale":"2023-06-28T20:07:23.787460+0000","last_undegraded":"2023-06-28T20:07:23.787460+0000","last_fullsized":"2023-06-28T20:07:23.787460+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:28:24.312050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.200081+0000","last_change":"2023-06-28T20:07:21.374603+0000","last_active":"2023-06-28T20:07:23.200081+0000","last_peered":"2023-06-28T20:07:23.200081+0000","last_clean":"2023-06-28T20:07:23.200081+0000","last_became_active":"2023-06-28T20:07:21.374276+0000","last_became_peered":"2023-06-28T20:07:21.374276+0000","last_unstale":"2023-06-28T20:07:23.200081+0000","last_undegraded":"2023-06-28T20:07:23.200081+0000","last_fullsized":"2023-06-28T20:07:23.200081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:42:24.558661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787168+0000","last_change":"2023-06-28T20:07:20.382290+0000","last_active":"2023-06-28T20:07:23.787168+0000","last_peered":"2023-06-28T20:07:23.787168+0000","last_clean":"2023-06-28T20:07:23.787168+0000","last_became_active":"2023-06-28T20:07:20.381991+0000","last_became_peered":"2023-06-28T20:07:20.381991+0000","last_unstale":"2023-06-28T20:07:23.787168+0000","last_undegraded":"2023-06-28T20:07:23.787168+0000","last_fullsized":"2023-06-28T20:07:23.787168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:56:30.648855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786827+0000","last_change":"2023-06-28T20:07:21.391299+0000","last_active":"2023-06-28T20:07:23.786827+0000","last_peered":"2023-06-28T20:07:23.786827+0000","last_clean":"2023-06-28T20:07:23.786827+0000","last_became_active":"2023-06-28T20:07:21.390957+0000","last_became_peered":"2023-06-28T20:07:21.390957+0000","last_unstale":"2023-06-28T20:07:23.786827+0000","last_undegraded":"2023-06-28T20:07:23.786827+0000","last_fullsized":"2023-06-28T20:07:23.786827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:20:40.315136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787523+0000","last_change":"2023-06-28T20:07:20.378108+0000","last_active":"2023-06-28T20:07:23.787523+0000","last_peered":"2023-06-28T20:07:23.787523+0000","last_clean":"2023-06-28T20:07:23.787523+0000","last_became_active":"2023-06-28T20:07:20.377812+0000","last_became_peered":"2023-06-28T20:07:20.377812+0000","last_unstale":"2023-06-28T20:07:23.787523+0000","last_undegraded":"2023-06-28T20:07:23.787523+0000","last_fullsized":"2023-06-28T20:07:23.787523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:19.349178+0000","last_clean_scrub_stamp":"2023-06-28T20:07:19.349178+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:27:05.931919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.199602+0000","last_change":"2023-06-28T20:07:21.384739+0000","last_active":"2023-06-28T20:07:23.199602+0000","last_peered":"2023-06-28T20:07:23.199602+0000","last_clean":"2023-06-28T20:07:23.199602+0000","last_became_active":"2023-06-28T20:07:21.384136+0000","last_became_peered":"2023-06-28T20:07:21.384136+0000","last_unstale":"2023-06-28T20:07:23.199602+0000","last_undegraded":"2023-06-28T20:07:23.199602+0000","last_fullsized":"2023-06-28T20:07:23.199602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:58:50.569412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:22.382228+0000","last_change":"2023-06-28T20:07:21.386188+0000","last_active":"2023-06-28T20:07:22.382228+0000","last_peered":"2023-06-28T20:07:22.382228+0000","last_clean":"2023-06-28T20:07:22.382228+0000","last_became_active":"2023-06-28T20:07:21.385171+0000","last_became_peered":"2023-06-28T20:07:21.385171+0000","last_unstale":"2023-06-28T20:07:22.382228+0000","last_undegraded":"2023-06-28T20:07:22.382228+0000","last_fullsized":"2023-06-28T20:07:22.382228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:43:18.447196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.786806+0000","last_change":"2023-06-28T20:07:21.402344+0000","last_active":"2023-06-28T20:07:23.786806+0000","last_peered":"2023-06-28T20:07:23.786806+0000","last_clean":"2023-06-28T20:07:23.786806+0000","last_became_active":"2023-06-28T20:07:21.402005+0000","last_became_peered":"2023-06-28T20:07:21.402005+0000","last_unstale":"2023-06-28T20:07:23.786806+0000","last_undegraded":"2023-06-28T20:07:23.786806+0000","last_fullsized":"2023-06-28T20:07:23.786806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:31:11.922720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.199807+0000","last_change":"2023-06-28T20:07:21.384643+0000","last_active":"2023-06-28T20:07:23.199807+0000","last_peered":"2023-06-28T20:07:23.199807+0000","last_clean":"2023-06-28T20:07:23.199807+0000","last_became_active":"2023-06-28T20:07:21.384232+0000","last_became_peered":"2023-06-28T20:07:21.384232+0000","last_unstale":"2023-06-28T20:07:23.199807+0000","last_undegraded":"2023-06-28T20:07:23.199807+0000","last_fullsized":"2023-06-28T20:07:23.199807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:56:31.654337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.787158+0000","last_change":"2023-06-28T20:07:21.391594+0000","last_active":"2023-06-28T20:07:23.787158+0000","last_peered":"2023-06-28T20:07:23.787158+0000","last_clean":"2023-06-28T20:07:23.787158+0000","last_became_active":"2023-06-28T20:07:21.391222+0000","last_became_peered":"2023-06-28T20:07:21.391222+0000","last_unstale":"2023-06-28T20:07:23.787158+0000","last_undegraded":"2023-06-28T20:07:23.787158+0000","last_fullsized":"2023-06-28T20:07:23.787158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:21:32.966251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-06-28T20:07:23.772192+0000","last_change":"2023-06-28T20:07:21.378640+0000","last_active":"2023-06-28T20:07:23.772192+0000","last_peered":"2023-06-28T20:07:23.772192+0000","last_clean":"2023-06-28T20:07:23.772192+0000","last_became_active":"2023-06-28T20:07:21.378345+0000","last_became_peered":"2023-06-28T20:07:21.378345+0000","last_unstale":"2023-06-28T20:07:23.772192+0000","last_undegraded":"2023-06-28T20:07:23.772192+0000","last_fullsized":"2023-06-28T20:07:23.772192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:20.354659+0000","last_clean_scrub_stamp":"2023-06-28T20:07:20.354659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:55:38.748243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":839680,"data_stored":671247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":79099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":811008,"data_stored":669467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":78720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":79099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":221184,"data_stored":79189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":80719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.0700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":225280,"data_stored":79751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-28T20:07:30.829 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-06-28T20:07:30.829 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-06-28T20:07:30.830 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-06-28T20:07:30.830 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-06-28T20:07:30.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.983+0000 7f3578d22700 1 Processor -- start 2023-06-28T20:07:30.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.983+0000 7f3578d22700 1 -- start start 2023-06-28T20:07:30.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.984+0000 7f3578d22700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3574070cb0 0x7f3574069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:30.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.984+0000 7f3578d22700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f357406a850 con 0x7f357406a430 2023-06-28T20:07:30.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.984+0000 7f3578d22700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3574071170 con 0x7f357415d9f0 2023-06-28T20:07:30.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.984+0000 7f3578d22700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f357415ddc0 con 0x7f3574070cb0 2023-06-28T20:07:30.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.984+0000 7f3572d9d700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f357406a430 0x7f35740cba40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42584/0 (socket says 172.21.15.43:42584) 2023-06-28T20:07:30.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.984+0000 7f3572d9d700 1 -- 172.21.15.43:0/4225746020 learned_addr learned my addr 172.21.15.43:0/4225746020 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:30.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.984+0000 7f3571d9b700 1 --2- 172.21.15.43:0/4225746020 >> v2:172.21.15.178:3300/0 conn(0x7f3574070cb0 0x7f3574069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:30.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.984+0000 7f357159a700 1 -- 172.21.15.43:0/4225746020 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596061839 0 0) 0x7f357406a850 con 0x7f357406a430 2023-06-28T20:07:30.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.984+0000 7f357159a700 1 -- 172.21.15.43:0/4225746020 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f355c0035b0 con 0x7f357406a430 2023-06-28T20:07:30.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.984+0000 7f357159a700 1 -- 172.21.15.43:0/4225746020 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2516646899 0 0) 0x7f355c0035b0 con 0x7f357406a430 2023-06-28T20:07:30.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.984+0000 7f357159a700 1 -- 172.21.15.43:0/4225746020 >> v2:172.21.15.178:3300/0 conn(0x7f3574070cb0 msgr2=0x7f3574069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:30.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.984+0000 7f357159a700 1 --2- 172.21.15.43:0/4225746020 >> v2:172.21.15.178:3300/0 conn(0x7f3574070cb0 0x7f3574069ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.984+0000 7f357159a700 1 -- 172.21.15.43:0/4225746020 >> v1:172.21.15.178:6789/0 conn(0x7f357415d9f0 legacy=0x7f357406d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:30.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.984+0000 7f357159a700 1 -- 172.21.15.43:0/4225746020 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3574071450 con 0x7f357406a430 2023-06-28T20:07:30.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.984+0000 7f357159a700 1 -- 172.21.15.43:0/4225746020 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f356c0019f0 con 0x7f357406a430 2023-06-28T20:07:30.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.984+0000 7f357159a700 1 -- 172.21.15.43:0/4225746020 >> v1:172.21.15.43:6789/0 conn(0x7f357406a430 legacy=0x7f35740cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:30.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.985+0000 7f357159a700 1 --2- 172.21.15.43:0/4225746020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f355c003a00 0x7f355c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:30.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.985+0000 7f357159a700 1 --2- 172.21.15.43:0/4225746020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f355c004850 0x7f355c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:30.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.985+0000 7f357159a700 1 --2- 172.21.15.43:0/4225746020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f355c005200 0x7f355c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:30.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.985+0000 7f357159a700 1 -- 172.21.15.43:0/4225746020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357415ddc0 con 0x7f355c005200 2023-06-28T20:07:30.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.985+0000 7f357159a700 1 -- 172.21.15.43:0/4225746020 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355c00c110 con 0x7f355c003a00 2023-06-28T20:07:30.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.985+0000 7f357159a700 1 -- 172.21.15.43:0/4225746020 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355c00c2d0 con 0x7f355c004850 2023-06-28T20:07:30.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.985+0000 7f3572d9d700 1 --2- 172.21.15.43:0/4225746020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f355c005200 0x7f355c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:30.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.985+0000 7f357259c700 1 --2- 172.21.15.43:0/4225746020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f355c003a00 0x7f355c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:30.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.985+0000 7f3571d9b700 1 --2- 172.21.15.43:0/4225746020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f355c004850 0x7f355c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:30.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.985+0000 7f3572d9d700 1 -- 172.21.15.43:0/4225746020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f355c004850 msgr2=0x7f355c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:30.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.985+0000 7f3572d9d700 1 --2- 172.21.15.43:0/4225746020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f355c004850 0x7f355c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.985+0000 7f3572d9d700 1 -- 172.21.15.43:0/4225746020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f355c003a00 msgr2=0x7f355c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:30.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.985+0000 7f3572d9d700 1 --2- 172.21.15.43:0/4225746020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f355c003a00 0x7f355c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.985+0000 7f3572d9d700 1 -- 172.21.15.43:0/4225746020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f355c0035b0 con 0x7f355c005200 2023-06-28T20:07:30.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.985+0000 7f3572d9d700 1 -- 172.21.15.43:0/4225746020 reap_dead start 2023-06-28T20:07:30.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.985+0000 7f3572d9d700 1 --2- 172.21.15.43:0/4225746020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f355c005200 0x7f355c00ba10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f356c00a830 tx=0x7f355c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f86e1f8629fac158 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:30.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.985+0000 7f357159a700 1 -- 172.21.15.43:0/4225746020 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c00a2f0 con 0x7f355c005200 2023-06-28T20:07:30.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.985+0000 7f357159a700 1 -- 172.21.15.43:0/4225746020 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356c012620 con 0x7f355c005200 2023-06-28T20:07:30.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.985+0000 7f357159a700 1 -- 172.21.15.43:0/4225746020 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c0129d0 con 0x7f355c005200 2023-06-28T20:07:30.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.986+0000 7f3578d22700 1 -- 172.21.15.43:0/4225746020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f355c005200 msgr2=0x7f355c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:30.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.986+0000 7f3578d22700 1 --2- 172.21.15.43:0/4225746020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f355c005200 0x7f355c00ba10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f356c00a830 tx=0x7f355c005880 comp rx=0 tx=0).stop 2023-06-28T20:07:30.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.986+0000 7f3578d22700 1 -- 172.21.15.43:0/4225746020 shutdown_connections 2023-06-28T20:07:30.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.986+0000 7f3578d22700 1 --2- 172.21.15.43:0/4225746020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f355c005200 0x7f355c00ba10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:30.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.986+0000 7f3578d22700 1 -- 172.21.15.43:0/4225746020 >> 172.21.15.43:0/4225746020 conn(0x7f35740b1720 msgr2=0x7f35740b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:30.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.986+0000 7f3578d22700 1 -- 172.21.15.43:0/4225746020 shutdown_connections 2023-06-28T20:07:30.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.986+0000 7f3578d22700 1 -- 172.21.15.43:0/4225746020 wait complete. 2023-06-28T20:07:30.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.988+0000 7f3578d22700 1 Processor -- start 2023-06-28T20:07:31.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.988+0000 7f3578d22700 1 -- start start 2023-06-28T20:07:31.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.989+0000 7f3578d22700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f355c005200 0x7f3574147120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:31.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.989+0000 7f3578d22700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35740c6ce0 0x7f357415c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:31.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.989+0000 7f3578d22700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f357415ca50 0x7f35740b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:31.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.989+0000 7f3578d22700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357415c080 con 0x7f35740c6ce0 2023-06-28T20:07:31.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.989+0000 7f3578d22700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357415b470 con 0x7f355c005200 2023-06-28T20:07:31.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.989+0000 7f3578d22700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357415b750 con 0x7f357415ca50 2023-06-28T20:07:31.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.989+0000 7f3571d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35740c6ce0 0x7f357415c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:31.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.989+0000 7f3571d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35740c6ce0 0x7f357415c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53410/0 (socket says 172.21.15.43:53410) 2023-06-28T20:07:31.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.989+0000 7f3571d9b700 1 -- 172.21.15.43:0/4105208663 learned_addr learned my addr 172.21.15.43:0/4105208663 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:31.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.989+0000 7f3572d9d700 1 --2- 172.21.15.43:0/4105208663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f357415ca50 0x7f35740b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:31.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.989+0000 7f357259c700 1 --2- 172.21.15.43:0/4105208663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f355c005200 0x7f3574147120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:31.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.989+0000 7f3571d9b700 1 -- 172.21.15.43:0/4105208663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f357415ca50 msgr2=0x7f35740b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:31.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.989+0000 7f3571d9b700 1 --2- 172.21.15.43:0/4105208663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f357415ca50 0x7f35740b4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.989+0000 7f3571d9b700 1 -- 172.21.15.43:0/4105208663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f355c005200 msgr2=0x7f3574147120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:31.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.989+0000 7f3571d9b700 1 --2- 172.21.15.43:0/4105208663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f355c005200 0x7f3574147120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.989+0000 7f3571d9b700 1 -- 172.21.15.43:0/4105208663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f357415ba30 con 0x7f35740c6ce0 2023-06-28T20:07:31.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.989+0000 7f357259c700 1 --2- 172.21.15.43:0/4105208663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f355c005200 0x7f3574147120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:31.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.989+0000 7f3571d9b700 1 --2- 172.21.15.43:0/4105208663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35740c6ce0 0x7f357415c510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3564011940 tx=0x7f3564011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:31.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.989+0000 7f3562ffd700 1 -- 172.21.15.43:0/4105208663 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3564011e90 con 0x7f35740c6ce0 2023-06-28T20:07:31.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.989+0000 7f3562ffd700 1 -- 172.21.15.43:0/4105208663 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3564004240 con 0x7f35740c6ce0 2023-06-28T20:07:31.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.990+0000 7f3578d22700 1 -- 172.21.15.43:0/4105208663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f357415bd10 con 0x7f35740c6ce0 2023-06-28T20:07:31.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.990+0000 7f3562ffd700 1 -- 172.21.15.43:0/4105208663 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356400f730 con 0x7f35740c6ce0 2023-06-28T20:07:31.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.990+0000 7f3578d22700 1 -- 172.21.15.43:0/4105208663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35741544b0 con 0x7f35740c6ce0 2023-06-28T20:07:31.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.991+0000 7f3578d22700 1 -- 172.21.15.43:0/4105208663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f357415c230 con 0x7f35740c6ce0 2023-06-28T20:07:31.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.991+0000 7f3562ffd700 1 -- 172.21.15.43:0/4105208663 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f356400f890 con 0x7f35740c6ce0 2023-06-28T20:07:31.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.992+0000 7f3562ffd700 1 --2- 172.21.15.43:0/4105208663 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f355406ad10 0x7f355406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:31.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.992+0000 7f3562ffd700 1 -- 172.21.15.43:0/4105208663 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3564089c40 con 0x7f35740c6ce0 2023-06-28T20:07:31.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.992+0000 7f357259c700 1 --2- 172.21.15.43:0/4105208663 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f355406ad10 0x7f355406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:31.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.992+0000 7f357259c700 1 --2- 172.21.15.43:0/4105208663 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f355406ad10 0x7f355406d1c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3574159380 tx=0x7f3568009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:31.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:30.996+0000 7f3562ffd700 1 -- 172.21.15.43:0/4105208663 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3564054d60 con 0x7f35740c6ce0 2023-06-28T20:07:31.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.240+0000 7f3578d22700 1 -- 172.21.15.43:0/4105208663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f357406f3b0 con 0x7f35740c6ce0 2023-06-28T20:07:31.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.240+0000 7f3562ffd700 1 -- 172.21.15.43:0/4105208663 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f3564058380 con 0x7f35740c6ce0 2023-06-28T20:07:31.244 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:31.245 INFO:teuthology.orchestra.run.smithi043.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-06-28T20:07:31.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.242+0000 7f3578d22700 1 -- 172.21.15.43:0/4105208663 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f355406ad10 msgr2=0x7f355406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:31.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.242+0000 7f3578d22700 1 --2- 172.21.15.43:0/4105208663 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f355406ad10 0x7f355406d1c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3574159380 tx=0x7f3568009040 comp rx=0 tx=0).stop 2023-06-28T20:07:31.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.242+0000 7f3578d22700 1 -- 172.21.15.43:0/4105208663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35740c6ce0 msgr2=0x7f357415c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:31.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.242+0000 7f3578d22700 1 --2- 172.21.15.43:0/4105208663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35740c6ce0 0x7f357415c510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3564011940 tx=0x7f3564011c50 comp rx=0 tx=0).stop 2023-06-28T20:07:31.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.242+0000 7f3578d22700 1 -- 172.21.15.43:0/4105208663 shutdown_connections 2023-06-28T20:07:31.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.242+0000 7f3578d22700 1 --2- 172.21.15.43:0/4105208663 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f355406ad10 0x7f355406d1c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.242+0000 7f3578d22700 1 --2- 172.21.15.43:0/4105208663 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f357415ca50 0x7f35740b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.242+0000 7f3578d22700 1 --2- 172.21.15.43:0/4105208663 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f355c005200 0x7f3574147120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.242+0000 7f3578d22700 1 --2- 172.21.15.43:0/4105208663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35740c6ce0 0x7f357415c510 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.242+0000 7f3578d22700 1 -- 172.21.15.43:0/4105208663 >> 172.21.15.43:0/4105208663 conn(0x7f35740b1720 msgr2=0x7f35740b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:31.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.243+0000 7f3578d22700 1 -- 172.21.15.43:0/4105208663 shutdown_connections 2023-06-28T20:07:31.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.243+0000 7f3578d22700 1 -- 172.21.15.43:0/4105208663 wait complete. 2023-06-28T20:07:31.258 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-06-28T20:07:31.259 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:07:31.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.382+0000 7f9e8f16d700 1 Processor -- start 2023-06-28T20:07:31.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.382+0000 7f9e8f16d700 1 -- start start 2023-06-28T20:07:31.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.383+0000 7f9e8f16d700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9e880cbd40 0x7f9e8806b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:31.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.383+0000 7f9e8f16d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9e8806b540 0x7f9e880ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:31.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.383+0000 7f9e8f16d700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e8806b9a0 con 0x7f9e880d0630 2023-06-28T20:07:31.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.383+0000 7f9e8f16d700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e880ccf10 con 0x7f9e8806b540 2023-06-28T20:07:31.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.383+0000 7f9e8f16d700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e8815cd80 con 0x7f9e880cbd40 2023-06-28T20:07:31.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.383+0000 7f9e8d70a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9e8806b540 0x7f9e880ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:31.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.383+0000 7f9e8cf09700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9e880d0630 0x7f9e880d1070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42602/0 (socket says 172.21.15.43:42602) 2023-06-28T20:07:31.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.383+0000 7f9e8cf09700 1 -- 172.21.15.43:0/2326573128 learned_addr learned my addr 172.21.15.43:0/2326573128 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:31.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.383+0000 7f9e87fff700 1 --2- 172.21.15.43:0/2326573128 >> v2:172.21.15.178:3300/0 conn(0x7f9e880cbd40 0x7f9e8806b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:31.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.383+0000 7f9e8d70a700 1 -- 172.21.15.43:0/2326573128 >> v2:172.21.15.178:3300/0 conn(0x7f9e880cbd40 msgr2=0x7f9e8806b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:31.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.383+0000 7f9e8d70a700 1 --2- 172.21.15.43:0/2326573128 >> v2:172.21.15.178:3300/0 conn(0x7f9e880cbd40 0x7f9e8806b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.383+0000 7f9e8d70a700 1 -- 172.21.15.43:0/2326573128 >> v1:172.21.15.43:6789/0 conn(0x7f9e880d0630 legacy=0x7f9e880d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:31.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.383+0000 7f9e8d70a700 1 -- 172.21.15.43:0/2326573128 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e880d0a00 con 0x7f9e8806b540 2023-06-28T20:07:31.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.383+0000 7f9e877fe700 1 -- 172.21.15.43:0/2326573128 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 852818808 0 0) 0x7f9e8806b9a0 con 0x7f9e880d0630 2023-06-28T20:07:31.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.383+0000 7f9e8d70a700 1 --2- 172.21.15.43:0/2326573128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e8806b540 0x7f9e880ce940 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009d90 tx=0x7f9e8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce0b54eb5bce9a5 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:31.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.384+0000 7f9e877fe700 1 -- 172.21.15.43:0/2326573128 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8000e070 con 0x7f9e8806b540 2023-06-28T20:07:31.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.384+0000 7f9e877fe700 1 -- 172.21.15.43:0/2326573128 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8000bae0 con 0x7f9e8806b540 2023-06-28T20:07:31.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.384+0000 7f9e877fe700 1 -- 172.21.15.43:0/2326573128 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8000be90 con 0x7f9e8806b540 2023-06-28T20:07:31.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.384+0000 7f9e8f16d700 1 -- 172.21.15.43:0/2326573128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e8806b540 msgr2=0x7f9e880ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:31.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.384+0000 7f9e8f16d700 1 --2- 172.21.15.43:0/2326573128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e8806b540 0x7f9e880ce940 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009d90 tx=0x7f9e8000b330 comp rx=0 tx=0).stop 2023-06-28T20:07:31.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.384+0000 7f9e8f16d700 1 -- 172.21.15.43:0/2326573128 shutdown_connections 2023-06-28T20:07:31.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.384+0000 7f9e8f16d700 1 --2- 172.21.15.43:0/2326573128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e8806b540 0x7f9e880ce940 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.384+0000 7f9e8f16d700 1 --2- 172.21.15.43:0/2326573128 >> v2:172.21.15.178:3300/0 conn(0x7f9e880cbd40 0x7f9e8806b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.384+0000 7f9e8f16d700 1 -- 172.21.15.43:0/2326573128 >> 172.21.15.43:0/2326573128 conn(0x7f9e880664b0 msgr2=0x7f9e8809a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:31.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.384+0000 7f9e8f16d700 1 -- 172.21.15.43:0/2326573128 shutdown_connections 2023-06-28T20:07:31.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.384+0000 7f9e8f16d700 1 -- 172.21.15.43:0/2326573128 wait complete. 2023-06-28T20:07:31.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.385+0000 7f9e8f16d700 1 Processor -- start 2023-06-28T20:07:31.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.385+0000 7f9e8f16d700 1 -- start start 2023-06-28T20:07:31.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.386+0000 7f9e8f16d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e8806b540 0x7f9e880b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:31.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.386+0000 7f9e8f16d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e880cbd40 0x7f9e881581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:31.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.386+0000 7f9e8f16d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e880d0630 0x7f9e88159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:31.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.386+0000 7f9e8f16d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e880b1420 con 0x7f9e880cbd40 2023-06-28T20:07:31.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.386+0000 7f9e8f16d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e880b1670 con 0x7f9e880d0630 2023-06-28T20:07:31.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.386+0000 7f9e8f16d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e880b1950 con 0x7f9e8806b540 2023-06-28T20:07:31.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.386+0000 7f9e87fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e880cbd40 0x7f9e881581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:31.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.386+0000 7f9e87fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e880cbd40 0x7f9e881581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53424/0 (socket says 172.21.15.43:53424) 2023-06-28T20:07:31.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.386+0000 7f9e87fff700 1 -- 172.21.15.43:0/3658257792 learned_addr learned my addr 172.21.15.43:0/3658257792 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:31.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.386+0000 7f9e8d70a700 1 --2- 172.21.15.43:0/3658257792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e880d0630 0x7f9e88159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:31.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.386+0000 7f9e8cf09700 1 --2- 172.21.15.43:0/3658257792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e8806b540 0x7f9e880b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:31.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.387+0000 7f9e87fff700 1 -- 172.21.15.43:0/3658257792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e8806b540 msgr2=0x7f9e880b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.387+0000 7f9e87fff700 1 --2- 172.21.15.43:0/3658257792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e8806b540 0x7f9e880b0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.387+0000 7f9e87fff700 1 -- 172.21.15.43:0/3658257792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e880d0630 msgr2=0x7f9e88159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.387+0000 7f9e87fff700 1 --2- 172.21.15.43:0/3658257792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e880d0630 0x7f9e88159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.387+0000 7f9e87fff700 1 -- 172.21.15.43:0/3658257792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e80009a70 con 0x7f9e880cbd40 2023-06-28T20:07:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.387+0000 7f9e87fff700 1 --2- 172.21.15.43:0/3658257792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e880cbd40 0x7f9e881581f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400e870 tx=0x7f9e7400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.387+0000 7f9e857fa700 1 -- 172.21.15.43:0/3658257792 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e74009590 con 0x7f9e880cbd40 2023-06-28T20:07:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.387+0000 7f9e857fa700 1 -- 172.21.15.43:0/3658257792 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e74013680 con 0x7f9e880cbd40 2023-06-28T20:07:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.387+0000 7f9e8f16d700 1 -- 172.21.15.43:0/3658257792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e880b1c30 con 0x7f9e880cbd40 2023-06-28T20:07:31.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.387+0000 7f9e857fa700 1 -- 172.21.15.43:0/3658257792 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e740139f0 con 0x7f9e880cbd40 2023-06-28T20:07:31.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.387+0000 7f9e8f16d700 1 -- 172.21.15.43:0/3658257792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e880b21f0 con 0x7f9e880cbd40 2023-06-28T20:07:31.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.388+0000 7f9e8f16d700 1 -- 172.21.15.43:0/3658257792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e880728b0 con 0x7f9e880cbd40 2023-06-28T20:07:31.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.388+0000 7f9e857fa700 1 -- 172.21.15.43:0/3658257792 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9e74013b50 con 0x7f9e880cbd40 2023-06-28T20:07:31.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.389+0000 7f9e857fa700 1 --2- 172.21.15.43:0/3658257792 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9e6c06ace0 0x7f9e6c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:31.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.389+0000 7f9e857fa700 1 -- 172.21.15.43:0/3658257792 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9e74089280 con 0x7f9e880cbd40 2023-06-28T20:07:31.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.389+0000 7f9e8cf09700 1 --2- 172.21.15.43:0/3658257792 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9e6c06ace0 0x7f9e6c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:31.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.389+0000 7f9e8cf09700 1 --2- 172.21.15.43:0/3658257792 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9e6c06ace0 0x7f9e6c06d190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9e881f5790 tx=0x7f9e7c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:31.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.393+0000 7f9e857fa700 1 -- 172.21.15.43:0/3658257792 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9e74057dc0 con 0x7f9e880cbd40 2023-06-28T20:07:31.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.596+0000 7f9e8f16d700 1 -- 172.21.15.43:0/3658257792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e880ccb90 con 0x7f9e880cbd40 2023-06-28T20:07:31.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.597+0000 7f9e857fa700 1 -- 172.21.15.43:0/3658257792 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18221 (secure 0 0 0) 0x7f9e74057950 con 0x7f9e880cbd40 2023-06-28T20:07:31.600 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:31.600 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:07:22.377248+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:07:18.624759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-28T20:07:19.760017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:07:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.599+0000 7f9e8f16d700 1 -- 172.21.15.43:0/3658257792 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9e6c06ace0 msgr2=0x7f9e6c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.599+0000 7f9e8f16d700 1 --2- 172.21.15.43:0/3658257792 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9e6c06ace0 0x7f9e6c06d190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9e881f5790 tx=0x7f9e7c009040 comp rx=0 tx=0).stop 2023-06-28T20:07:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.599+0000 7f9e8f16d700 1 -- 172.21.15.43:0/3658257792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e880cbd40 msgr2=0x7f9e881581f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.599+0000 7f9e8f16d700 1 --2- 172.21.15.43:0/3658257792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e880cbd40 0x7f9e881581f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400e870 tx=0x7f9e7400eb80 comp rx=0 tx=0).stop 2023-06-28T20:07:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.599+0000 7f9e8f16d700 1 -- 172.21.15.43:0/3658257792 shutdown_connections 2023-06-28T20:07:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.599+0000 7f9e8f16d700 1 --2- 172.21.15.43:0/3658257792 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e880d0630 0x7f9e88159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.599+0000 7f9e8f16d700 1 --2- 172.21.15.43:0/3658257792 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9e6c06ace0 0x7f9e6c06d190 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.599+0000 7f9e8f16d700 1 --2- 172.21.15.43:0/3658257792 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9e8806b540 0x7f9e880b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.599+0000 7f9e8f16d700 1 --2- 172.21.15.43:0/3658257792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e880cbd40 0x7f9e881581f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.599+0000 7f9e8f16d700 1 -- 172.21.15.43:0/3658257792 >> 172.21.15.43:0/3658257792 conn(0x7f9e880664b0 msgr2=0x7f9e880cd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:31.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.599+0000 7f9e8f16d700 1 -- 172.21.15.43:0/3658257792 shutdown_connections 2023-06-28T20:07:31.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.599+0000 7f9e8f16d700 1 -- 172.21.15.43:0/3658257792 wait complete. 2023-06-28T20:07:31.614 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-28T20:07:13.580969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-28T20:07:14.038358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-28T20:07:18.624759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-28T20:07:19.760017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-28T20:07:31.615 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-28T20:07:31.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.739+0000 7fda18302700 1 Processor -- start 2023-06-28T20:07:31.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.739+0000 7fda18302700 1 -- start start 2023-06-28T20:07:31.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.740+0000 7fda18302700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fda100caff0 0x7fda1006b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:31.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.740+0000 7fda18302700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fda100cb3c0 0x7fda1006c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:31.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.740+0000 7fda18302700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda1006a020 con 0x7fda1006c5e0 2023-06-28T20:07:31.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.740+0000 7fda18302700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda1006ca60 con 0x7fda100caff0 2023-06-28T20:07:31.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.740+0000 7fda18302700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda1006a300 con 0x7fda100cb3c0 2023-06-28T20:07:31.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.740+0000 7fda1609e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fda100caff0 0x7fda1006b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:31.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.741+0000 7fda1609e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fda100caff0 0x7fda1006b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53430/0 (socket says 172.21.15.43:53430) 2023-06-28T20:07:31.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.741+0000 7fda1609e700 1 -- 172.21.15.43:0/2303050682 learned_addr learned my addr 172.21.15.43:0/2303050682 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:31.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.741+0000 7fda1589d700 1 --2- 172.21.15.43:0/2303050682 >> v2:172.21.15.178:3300/0 conn(0x7fda100cb3c0 0x7fda1006c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:31.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.741+0000 7fda1609e700 1 -- 172.21.15.43:0/2303050682 >> v2:172.21.15.178:3300/0 conn(0x7fda100cb3c0 msgr2=0x7fda1006c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:31.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.741+0000 7fda1609e700 1 --2- 172.21.15.43:0/2303050682 >> v2:172.21.15.178:3300/0 conn(0x7fda100cb3c0 0x7fda1006c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.741+0000 7fda1609e700 1 -- 172.21.15.43:0/2303050682 >> v1:172.21.15.178:6789/0 conn(0x7fda1006c5e0 legacy=0x7fda1015df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:31.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.741+0000 7fda1609e700 1 -- 172.21.15.43:0/2303050682 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1006a5e0 con 0x7fda100caff0 2023-06-28T20:07:31.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.741+0000 7fda1589d700 1 --2- 172.21.15.43:0/2303050682 >> v2:172.21.15.178:3300/0 conn(0x7fda100cb3c0 0x7fda1006c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:31.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.741+0000 7fda1609e700 1 --2- 172.21.15.43:0/2303050682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda100caff0 0x7fda1006b230 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0098f0 tx=0x7fda0c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a1783a01362c72 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:31.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.741+0000 7fda1509c700 1 -- 172.21.15.43:0/2303050682 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c00b860 con 0x7fda100caff0 2023-06-28T20:07:31.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.741+0000 7fda1509c700 1 -- 172.21.15.43:0/2303050682 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0c00b9c0 con 0x7fda100caff0 2023-06-28T20:07:31.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.741+0000 7fda1509c700 1 -- 172.21.15.43:0/2303050682 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c00bd50 con 0x7fda100caff0 2023-06-28T20:07:31.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.742+0000 7fda18302700 1 -- 172.21.15.43:0/2303050682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda100caff0 msgr2=0x7fda1006b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:31.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.742+0000 7fda18302700 1 --2- 172.21.15.43:0/2303050682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda100caff0 0x7fda1006b230 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0098f0 tx=0x7fda0c009c00 comp rx=0 tx=0).stop 2023-06-28T20:07:31.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.742+0000 7fda18302700 1 -- 172.21.15.43:0/2303050682 shutdown_connections 2023-06-28T20:07:31.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.742+0000 7fda18302700 1 --2- 172.21.15.43:0/2303050682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda100caff0 0x7fda1006b230 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.742+0000 7fda18302700 1 --2- 172.21.15.43:0/2303050682 >> v2:172.21.15.178:3300/0 conn(0x7fda100cb3c0 0x7fda1006c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.742+0000 7fda18302700 1 -- 172.21.15.43:0/2303050682 >> 172.21.15.43:0/2303050682 conn(0x7fda100b1a50 msgr2=0x7fda100cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:31.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.742+0000 7fda18302700 1 -- 172.21.15.43:0/2303050682 shutdown_connections 2023-06-28T20:07:31.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.742+0000 7fda18302700 1 -- 172.21.15.43:0/2303050682 wait complete. 2023-06-28T20:07:31.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.743+0000 7fda18302700 1 Processor -- start 2023-06-28T20:07:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.743+0000 7fda18302700 1 -- start start 2023-06-28T20:07:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.744+0000 7fda18302700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda1006c5e0 0x7fda10151700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.744+0000 7fda18302700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda100caff0 0x7fda100c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.744+0000 7fda18302700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda100cb3c0 0x7fda100b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.744+0000 7fda18302700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda10152e10 con 0x7fda100caff0 2023-06-28T20:07:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.744+0000 7fda18302700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda100c4b30 con 0x7fda100cb3c0 2023-06-28T20:07:31.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.744+0000 7fda18302700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda100c4290 con 0x7fda1006c5e0 2023-06-28T20:07:31.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.744+0000 7fda1589d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda100caff0 0x7fda100c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:31.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.744+0000 7fda1589d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda100caff0 0x7fda100c37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53438/0 (socket says 172.21.15.43:53438) 2023-06-28T20:07:31.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.744+0000 7fda1589d700 1 -- 172.21.15.43:0/2202516148 learned_addr learned my addr 172.21.15.43:0/2202516148 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:31.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.744+0000 7fda1609e700 1 --2- 172.21.15.43:0/2202516148 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda1006c5e0 0x7fda10151700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:31.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.744+0000 7fda1689f700 1 --2- 172.21.15.43:0/2202516148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda100cb3c0 0x7fda100b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:31.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.744+0000 7fda1589d700 1 -- 172.21.15.43:0/2202516148 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda1006c5e0 msgr2=0x7fda10151700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:31.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.744+0000 7fda1589d700 1 --2- 172.21.15.43:0/2202516148 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda1006c5e0 0x7fda10151700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.744+0000 7fda1589d700 1 -- 172.21.15.43:0/2202516148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda100cb3c0 msgr2=0x7fda100b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:31.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.744+0000 7fda1589d700 1 --2- 172.21.15.43:0/2202516148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda100cb3c0 0x7fda100b3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.744+0000 7fda1589d700 1 -- 172.21.15.43:0/2202516148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0c009550 con 0x7fda100caff0 2023-06-28T20:07:31.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.744+0000 7fda1589d700 1 --2- 172.21.15.43:0/2202516148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda100caff0 0x7fda100c37f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fda0400e7e0 tx=0x7fda0400eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.744+0000 7fda02ffd700 1 -- 172.21.15.43:0/2202516148 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda04004230 con 0x7fda100caff0 2023-06-28T20:07:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.745+0000 7fda02ffd700 1 -- 172.21.15.43:0/2202516148 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda04004390 con 0x7fda100caff0 2023-06-28T20:07:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.745+0000 7fda18302700 1 -- 172.21.15.43:0/2202516148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda100c4570 con 0x7fda100caff0 2023-06-28T20:07:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.745+0000 7fda02ffd700 1 -- 172.21.15.43:0/2202516148 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda040036a0 con 0x7fda100caff0 2023-06-28T20:07:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.745+0000 7fda18302700 1 -- 172.21.15.43:0/2202516148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda10153040 con 0x7fda100caff0 2023-06-28T20:07:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.745+0000 7fda18302700 1 -- 172.21.15.43:0/2202516148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda10153260 con 0x7fda100caff0 2023-06-28T20:07:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.746+0000 7fda02ffd700 1 -- 172.21.15.43:0/2202516148 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fda04003800 con 0x7fda100caff0 2023-06-28T20:07:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.746+0000 7fda02ffd700 1 --2- 172.21.15.43:0/2202516148 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd9f406ace0 0x7fd9f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.746+0000 7fda02ffd700 1 -- 172.21.15.43:0/2202516148 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fda04088f80 con 0x7fda100caff0 2023-06-28T20:07:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.746+0000 7fda1609e700 1 --2- 172.21.15.43:0/2202516148 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd9f406ace0 0x7fd9f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.747+0000 7fda1609e700 1 --2- 172.21.15.43:0/2202516148 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd9f406ace0 0x7fd9f406d190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009c40 tx=0x7fda0c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.751+0000 7fda02ffd700 1 -- 172.21.15.43:0/2202516148 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fda04054030 con 0x7fda100caff0 2023-06-28T20:07:31.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.949+0000 7fda18302700 1 -- 172.21.15.43:0/2202516148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fda100728b0 con 0x7fda100caff0 2023-06-28T20:07:31.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.949+0000 7fda02ffd700 1 -- 172.21.15.43:0/2202516148 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fda04057650 con 0x7fda100caff0 2023-06-28T20:07:31.953 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-06-28T20:07:31.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.951+0000 7fda18302700 1 -- 172.21.15.43:0/2202516148 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd9f406ace0 msgr2=0x7fd9f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:31.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.951+0000 7fda18302700 1 --2- 172.21.15.43:0/2202516148 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd9f406ace0 0x7fd9f406d190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fda0c009c40 tx=0x7fda0c008030 comp rx=0 tx=0).stop 2023-06-28T20:07:31.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.951+0000 7fda18302700 1 -- 172.21.15.43:0/2202516148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda100caff0 msgr2=0x7fda100c37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:31.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.951+0000 7fda18302700 1 --2- 172.21.15.43:0/2202516148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda100caff0 0x7fda100c37f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fda0400e7e0 tx=0x7fda0400eba0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.951+0000 7fda18302700 1 -- 172.21.15.43:0/2202516148 shutdown_connections 2023-06-28T20:07:31.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.951+0000 7fda18302700 1 --2- 172.21.15.43:0/2202516148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fda100cb3c0 0x7fda100b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.951+0000 7fda18302700 1 --2- 172.21.15.43:0/2202516148 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd9f406ace0 0x7fd9f406d190 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.951+0000 7fda18302700 1 --2- 172.21.15.43:0/2202516148 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fda1006c5e0 0x7fda10151700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.951+0000 7fda18302700 1 --2- 172.21.15.43:0/2202516148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda100caff0 0x7fda100c37f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:31.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.951+0000 7fda18302700 1 -- 172.21.15.43:0/2202516148 >> 172.21.15.43:0/2202516148 conn(0x7fda100b1a50 msgr2=0x7fda10072ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:31.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.951+0000 7fda18302700 1 -- 172.21.15.43:0/2202516148 shutdown_connections 2023-06-28T20:07:31.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:31.951+0000 7fda18302700 1 -- 172.21.15.43:0/2202516148 wait complete. 2023-06-28T20:07:31.966 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-28T20:07:32.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.093+0000 7fcf38942700 1 Processor -- start 2023-06-28T20:07:32.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.093+0000 7fcf38942700 1 -- start start 2023-06-28T20:07:32.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.094+0000 7fcf38942700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf3415a5f0 0x7fcf340ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:32.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.094+0000 7fcf38942700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcf3406e320 0x7fcf340ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:32.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.094+0000 7fcf38942700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf340ae950 con 0x7fcf340ad7d0 2023-06-28T20:07:32.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.094+0000 7fcf38942700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf3415a9c0 con 0x7fcf3406e320 2023-06-28T20:07:32.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.094+0000 7fcf38942700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf340add40 con 0x7fcf3415a5f0 2023-06-28T20:07:32.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.094+0000 7fcf31d9b700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcf340ad7d0 0x7fcf340aede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42632/0 (socket says 172.21.15.43:42632) 2023-06-28T20:07:32.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.094+0000 7fcf31d9b700 1 -- 172.21.15.43:0/2794654718 learned_addr learned my addr 172.21.15.43:0/2794654718 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:32.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.094+0000 7fcf32d9d700 1 --2- 172.21.15.43:0/2794654718 >> v2:172.21.15.178:3300/0 conn(0x7fcf3406e320 0x7fcf340ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:32.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.094+0000 7fcf3259c700 1 --2- 172.21.15.43:0/2794654718 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf3415a5f0 0x7fcf340ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:32.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.094+0000 7fcf3159a700 1 -- 172.21.15.43:0/2794654718 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570067749 0 0) 0x7fcf340ae950 con 0x7fcf340ad7d0 2023-06-28T20:07:32.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.094+0000 7fcf3159a700 1 -- 172.21.15.43:0/2794654718 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf180035b0 con 0x7fcf340ad7d0 2023-06-28T20:07:32.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.094+0000 7fcf3159a700 1 -- 172.21.15.43:0/2794654718 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2378897089 0 0) 0x7fcf180035b0 con 0x7fcf340ad7d0 2023-06-28T20:07:32.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.094+0000 7fcf3159a700 1 -- 172.21.15.43:0/2794654718 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf3415a5f0 msgr2=0x7fcf340ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:32.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.094+0000 7fcf3159a700 1 --2- 172.21.15.43:0/2794654718 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf3415a5f0 0x7fcf340ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.094+0000 7fcf3159a700 1 -- 172.21.15.43:0/2794654718 >> v2:172.21.15.178:3300/0 conn(0x7fcf3406e320 msgr2=0x7fcf340ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:32.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.094+0000 7fcf3159a700 1 --2- 172.21.15.43:0/2794654718 >> v2:172.21.15.178:3300/0 conn(0x7fcf3406e320 0x7fcf340ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.094+0000 7fcf3159a700 1 -- 172.21.15.43:0/2794654718 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf340ae020 con 0x7fcf340ad7d0 2023-06-28T20:07:32.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.094+0000 7fcf3159a700 1 -- 172.21.15.43:0/2794654718 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7fcf24001670 con 0x7fcf340ad7d0 2023-06-28T20:07:32.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.095+0000 7fcf3159a700 1 -- 172.21.15.43:0/2794654718 >> v1:172.21.15.43:6789/0 conn(0x7fcf340ad7d0 legacy=0x7fcf340aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:32.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.095+0000 7fcf3159a700 1 --2- 172.21.15.43:0/2794654718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf18003a00 0x7fcf18003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:32.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.095+0000 7fcf3159a700 1 --2- 172.21.15.43:0/2794654718 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf3415a5f0 0x7fcf18004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:32.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.095+0000 7fcf3159a700 1 --2- 172.21.15.43:0/2794654718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf18004d90 0x7fcf18005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:32.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.095+0000 7fcf3159a700 1 -- 172.21.15.43:0/2794654718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3415a9c0 con 0x7fcf18004d90 2023-06-28T20:07:32.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.095+0000 7fcf3159a700 1 -- 172.21.15.43:0/2794654718 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf340add40 con 0x7fcf18003a00 2023-06-28T20:07:32.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.095+0000 7fcf3159a700 1 -- 172.21.15.43:0/2794654718 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1800ba10 con 0x7fcf3415a5f0 2023-06-28T20:07:32.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.095+0000 7fcf32d9d700 1 --2- 172.21.15.43:0/2794654718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf18004d90 0x7fcf18005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:32.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.095+0000 7fcf31d9b700 1 --2- 172.21.15.43:0/2794654718 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf3415a5f0 0x7fcf18004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:32.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.095+0000 7fcf3259c700 1 --2- 172.21.15.43:0/2794654718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf18003a00 0x7fcf18003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:32.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.095+0000 7fcf32d9d700 1 -- 172.21.15.43:0/2794654718 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf3415a5f0 msgr2=0x7fcf18004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:32.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.095+0000 7fcf32d9d700 1 --2- 172.21.15.43:0/2794654718 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf3415a5f0 0x7fcf18004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.095+0000 7fcf32d9d700 1 -- 172.21.15.43:0/2794654718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf18003a00 msgr2=0x7fcf18003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:32.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.095+0000 7fcf32d9d700 1 --2- 172.21.15.43:0/2794654718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf18003a00 0x7fcf18003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.095+0000 7fcf32d9d700 1 -- 172.21.15.43:0/2794654718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf180035b0 con 0x7fcf18004d90 2023-06-28T20:07:32.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.095+0000 7fcf32d9d700 1 --2- 172.21.15.43:0/2794654718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf18004d90 0x7fcf18005200 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009fc0 tx=0x7fcf18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e773300cdb9758 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:32.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.095+0000 7fcf3159a700 1 -- 172.21.15.43:0/2794654718 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c00ec30 con 0x7fcf18004d90 2023-06-28T20:07:32.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.095+0000 7fcf3159a700 1 -- 172.21.15.43:0/2794654718 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2c00f040 con 0x7fcf18004d90 2023-06-28T20:07:32.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.095+0000 7fcf3159a700 1 -- 172.21.15.43:0/2794654718 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c014480 con 0x7fcf18004d90 2023-06-28T20:07:32.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.096+0000 7fcf38942700 1 -- 172.21.15.43:0/2794654718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf18004d90 msgr2=0x7fcf18005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:32.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.096+0000 7fcf38942700 1 --2- 172.21.15.43:0/2794654718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf18004d90 0x7fcf18005200 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009fc0 tx=0x7fcf18005880 comp rx=0 tx=0).stop 2023-06-28T20:07:32.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.096+0000 7fcf32d9d700 1 -- 172.21.15.43:0/2794654718 reap_dead start 2023-06-28T20:07:32.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.096+0000 7fcf38942700 1 -- 172.21.15.43:0/2794654718 shutdown_connections 2023-06-28T20:07:32.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.096+0000 7fcf38942700 1 -- 172.21.15.43:0/2794654718 >> 172.21.15.43:0/2794654718 conn(0x7fcf340664b0 msgr2=0x7fcf340d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:32.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.096+0000 7fcf38942700 1 -- 172.21.15.43:0/2794654718 shutdown_connections 2023-06-28T20:07:32.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.096+0000 7fcf38942700 1 -- 172.21.15.43:0/2794654718 wait complete. 2023-06-28T20:07:32.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.097+0000 7fcf38942700 1 Processor -- start 2023-06-28T20:07:32.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.097+0000 7fcf38942700 1 -- start start 2023-06-28T20:07:32.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.098+0000 7fcf38942700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf18004d90 0x7fcf3414f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:32.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.098+0000 7fcf38942700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf3414fb80 0x7fcf340b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:32.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.098+0000 7fcf38942700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf340b2bc0 0x7fcf340cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:32.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.098+0000 7fcf38942700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf34156ef0 con 0x7fcf18004d90 2023-06-28T20:07:32.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.098+0000 7fcf38942700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf341571d0 con 0x7fcf340b2bc0 2023-06-28T20:07:32.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.098+0000 7fcf38942700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf340b1e80 con 0x7fcf3414fb80 2023-06-28T20:07:32.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.098+0000 7fcf3259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf18004d90 0x7fcf3414f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:32.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.098+0000 7fcf3259c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf18004d90 0x7fcf3414f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53456/0 (socket says 172.21.15.43:53456) 2023-06-28T20:07:32.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.098+0000 7fcf3259c700 1 -- 172.21.15.43:0/3282588899 learned_addr learned my addr 172.21.15.43:0/3282588899 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:32.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.098+0000 7fcf31d9b700 1 --2- 172.21.15.43:0/3282588899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf3414fb80 0x7fcf340b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:32.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.098+0000 7fcf32d9d700 1 --2- 172.21.15.43:0/3282588899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf340b2bc0 0x7fcf340cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:32.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.098+0000 7fcf3259c700 1 -- 172.21.15.43:0/3282588899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf3414fb80 msgr2=0x7fcf340b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:32.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.098+0000 7fcf3259c700 1 --2- 172.21.15.43:0/3282588899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf3414fb80 0x7fcf340b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.098+0000 7fcf3259c700 1 -- 172.21.15.43:0/3282588899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf340b2bc0 msgr2=0x7fcf340cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:32.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.098+0000 7fcf3259c700 1 --2- 172.21.15.43:0/3282588899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf340b2bc0 0x7fcf340cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.099+0000 7fcf3259c700 1 -- 172.21.15.43:0/3282588899 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf340b2160 con 0x7fcf18004d90 2023-06-28T20:07:32.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.099+0000 7fcf31d9b700 1 --2- 172.21.15.43:0/3282588899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf3414fb80 0x7fcf340b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:32.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.099+0000 7fcf32d9d700 1 --2- 172.21.15.43:0/3282588899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf340b2bc0 0x7fcf340cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:32.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.099+0000 7fcf3259c700 1 --2- 172.21.15.43:0/3282588899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf18004d90 0x7fcf3414f310 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcf200119f0 tx=0x7fcf20011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:32.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.099+0000 7fcf2affd700 1 -- 172.21.15.43:0/3282588899 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf20005c30 con 0x7fcf18004d90 2023-06-28T20:07:32.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.099+0000 7fcf2affd700 1 -- 172.21.15.43:0/3282588899 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf20005d90 con 0x7fcf18004d90 2023-06-28T20:07:32.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.099+0000 7fcf2affd700 1 -- 172.21.15.43:0/3282588899 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2000f950 con 0x7fcf18004d90 2023-06-28T20:07:32.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.099+0000 7fcf38942700 1 -- 172.21.15.43:0/3282588899 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf34156b80 con 0x7fcf18004d90 2023-06-28T20:07:32.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.099+0000 7fcf38942700 1 -- 172.21.15.43:0/3282588899 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf340b1aa0 con 0x7fcf18004d90 2023-06-28T20:07:32.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.100+0000 7fcf38942700 1 -- 172.21.15.43:0/3282588899 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf34157030 con 0x7fcf18004d90 2023-06-28T20:07:32.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.100+0000 7fcf2affd700 1 -- 172.21.15.43:0/3282588899 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcf2000fab0 con 0x7fcf18004d90 2023-06-28T20:07:32.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.101+0000 7fcf2affd700 1 --2- 172.21.15.43:0/3282588899 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fcf1c06ace0 0x7fcf1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:32.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.101+0000 7fcf31d9b700 1 --2- 172.21.15.43:0/3282588899 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fcf1c06ace0 0x7fcf1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:32.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.101+0000 7fcf2affd700 1 -- 172.21.15.43:0/3282588899 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcf20089c50 con 0x7fcf18004d90 2023-06-28T20:07:32.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.101+0000 7fcf31d9b700 1 --2- 172.21.15.43:0/3282588899 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fcf1c06ace0 0x7fcf1c06d190 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcf240015f0 tx=0x7fcf24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:32.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.106+0000 7fcf2affd700 1 -- 172.21.15.43:0/3282588899 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcf34157030 con 0x7fcf18004d90 2023-06-28T20:07:32.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.302+0000 7fcf38942700 1 -- 172.21.15.43:0/3282588899 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcf340b24d0 con 0x7fcf18004d90 2023-06-28T20:07:32.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.303+0000 7fcf2affd700 1 -- 172.21.15.43:0/3282588899 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcf200210a0 con 0x7fcf18004d90 2023-06-28T20:07:32.306 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-06-28T20:07:32.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.305+0000 7fcf38942700 1 -- 172.21.15.43:0/3282588899 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fcf1c06ace0 msgr2=0x7fcf1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:32.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.305+0000 7fcf38942700 1 --2- 172.21.15.43:0/3282588899 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fcf1c06ace0 0x7fcf1c06d190 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcf240015f0 tx=0x7fcf24009040 comp rx=0 tx=0).stop 2023-06-28T20:07:32.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.305+0000 7fcf38942700 1 -- 172.21.15.43:0/3282588899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf18004d90 msgr2=0x7fcf3414f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:32.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.305+0000 7fcf38942700 1 --2- 172.21.15.43:0/3282588899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf18004d90 0x7fcf3414f310 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcf200119f0 tx=0x7fcf20011db0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.305+0000 7fcf38942700 1 -- 172.21.15.43:0/3282588899 shutdown_connections 2023-06-28T20:07:32.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.305+0000 7fcf38942700 1 --2- 172.21.15.43:0/3282588899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf340b2bc0 0x7fcf340cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.305+0000 7fcf38942700 1 --2- 172.21.15.43:0/3282588899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf18004d90 0x7fcf3414f310 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.305+0000 7fcf38942700 1 --2- 172.21.15.43:0/3282588899 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fcf1c06ace0 0x7fcf1c06d190 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.305+0000 7fcf38942700 1 --2- 172.21.15.43:0/3282588899 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcf3414fb80 0x7fcf340b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.305+0000 7fcf38942700 1 -- 172.21.15.43:0/3282588899 >> 172.21.15.43:0/3282588899 conn(0x7fcf340664b0 msgr2=0x7fcf34159410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:32.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.305+0000 7fcf38942700 1 -- 172.21.15.43:0/3282588899 shutdown_connections 2023-06-28T20:07:32.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.305+0000 7fcf38942700 1 -- 172.21.15.43:0/3282588899 wait complete. 2023-06-28T20:07:32.320 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-28T20:07:32.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.445+0000 7f5ca30f5700 1 Processor -- start 2023-06-28T20:07:32.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.445+0000 7f5ca30f5700 1 -- start start 2023-06-28T20:07:32.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.446+0000 7f5ca30f5700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5c9c0aec90 0x7f5c9c160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:32.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.446+0000 7f5ca30f5700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c9c06ccc0 0x7f5c9c0b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:32.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.446+0000 7f5ca30f5700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c9c160250 con 0x7f5c9c0af060 2023-06-28T20:07:32.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.446+0000 7f5ca30f5700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c9c15cb30 con 0x7f5c9c06ccc0 2023-06-28T20:07:32.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.446+0000 7f5ca30f5700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c9c071410 con 0x7f5c9c0aec90 2023-06-28T20:07:32.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.446+0000 7f5ca1692700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c9c06ccc0 0x7f5c9c0b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:32.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.446+0000 7f5c9bfff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5c9c0af060 0x7f5c9c070bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42652/0 (socket says 172.21.15.43:42652) 2023-06-28T20:07:32.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.446+0000 7f5c9bfff700 1 -- 172.21.15.43:0/2480921297 learned_addr learned my addr 172.21.15.43:0/2480921297 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:32.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.446+0000 7f5ca1692700 1 --2- 172.21.15.43:0/2480921297 >> v2:172.21.15.43:3300/0 conn(0x7f5c9c06ccc0 0x7f5c9c0b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53468/0 (socket says 172.21.15.43:53468) 2023-06-28T20:07:32.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.446+0000 7f5ca1692700 1 -- 172.21.15.43:0/2480921297 >> v2:172.21.15.178:3300/0 conn(0x7f5c9c0aec90 msgr2=0x7f5c9c160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:32.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.446+0000 7f5ca0e91700 1 --2- 172.21.15.43:0/2480921297 >> v2:172.21.15.178:3300/0 conn(0x7f5c9c0aec90 0x7f5c9c160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:32.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.447+0000 7f5ca1692700 1 --2- 172.21.15.43:0/2480921297 >> v2:172.21.15.178:3300/0 conn(0x7f5c9c0aec90 0x7f5c9c160cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.447+0000 7f5ca1692700 1 -- 172.21.15.43:0/2480921297 >> v1:172.21.15.43:6789/0 conn(0x7f5c9c0af060 legacy=0x7f5c9c070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:32.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.447+0000 7f5ca1692700 1 -- 172.21.15.43:0/2480921297 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c1604f0 con 0x7f5c9c06ccc0 2023-06-28T20:07:32.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.447+0000 7f5c9b7fe700 1 -- 172.21.15.43:0/2480921297 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3746756684 0 0) 0x7f5c9c160250 con 0x7f5c9c0af060 2023-06-28T20:07:32.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.447+0000 7f5ca1692700 1 --2- 172.21.15.43:0/2480921297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c9c06ccc0 0x7f5c9c0b1880 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c06fce0 tx=0x7f5c9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33056fc787c61dcd server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:32.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.447+0000 7f5c9b7fe700 1 -- 172.21.15.43:0/2480921297 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9400e070 con 0x7f5c9c06ccc0 2023-06-28T20:07:32.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.447+0000 7f5c9b7fe700 1 -- 172.21.15.43:0/2480921297 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9400bbd0 con 0x7f5c9c06ccc0 2023-06-28T20:07:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.447+0000 7f5c9b7fe700 1 -- 172.21.15.43:0/2480921297 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94012430 con 0x7f5c9c06ccc0 2023-06-28T20:07:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.447+0000 7f5ca30f5700 1 -- 172.21.15.43:0/2480921297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c9c06ccc0 msgr2=0x7f5c9c0b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.447+0000 7f5ca30f5700 1 --2- 172.21.15.43:0/2480921297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c9c06ccc0 0x7f5c9c0b1880 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c06fce0 tx=0x7f5c9400b330 comp rx=0 tx=0).stop 2023-06-28T20:07:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.447+0000 7f5ca30f5700 1 -- 172.21.15.43:0/2480921297 shutdown_connections 2023-06-28T20:07:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.447+0000 7f5ca30f5700 1 --2- 172.21.15.43:0/2480921297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c9c06ccc0 0x7f5c9c0b1880 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.447+0000 7f5ca30f5700 1 --2- 172.21.15.43:0/2480921297 >> v2:172.21.15.178:3300/0 conn(0x7f5c9c0aec90 0x7f5c9c160cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.447+0000 7f5ca30f5700 1 -- 172.21.15.43:0/2480921297 >> 172.21.15.43:0/2480921297 conn(0x7f5c9c06b000 msgr2=0x7f5c9c06b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.447+0000 7f5ca30f5700 1 -- 172.21.15.43:0/2480921297 shutdown_connections 2023-06-28T20:07:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.447+0000 7f5ca30f5700 1 -- 172.21.15.43:0/2480921297 wait complete. 2023-06-28T20:07:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.449+0000 7f5ca30f5700 1 Processor -- start 2023-06-28T20:07:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.449+0000 7f5ca30f5700 1 -- start start 2023-06-28T20:07:32.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.449+0000 7f5ca30f5700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c9c06ccc0 0x7f5c9c0acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:32.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.449+0000 7f5ca30f5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c9c0aec90 0x7f5c9c0ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:32.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.450+0000 7f5ca30f5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c9c0af060 0x7f5c9c1538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:32.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.450+0000 7f5ca30f5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c157ec0 con 0x7f5c9c0aec90 2023-06-28T20:07:32.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.450+0000 7f5ca30f5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c1581a0 con 0x7f5c9c0af060 2023-06-28T20:07:32.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.450+0000 7f5ca30f5700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c158480 con 0x7f5c9c06ccc0 2023-06-28T20:07:32.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.450+0000 7f5c9bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c9c0aec90 0x7f5c9c0ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:32.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.450+0000 7f5c9bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c9c0aec90 0x7f5c9c0ad420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53470/0 (socket says 172.21.15.43:53470) 2023-06-28T20:07:32.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.450+0000 7f5c9bfff700 1 -- 172.21.15.43:0/3588375314 learned_addr learned my addr 172.21.15.43:0/3588375314 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:32.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.450+0000 7f5ca1692700 1 --2- 172.21.15.43:0/3588375314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c9c0af060 0x7f5c9c1538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:32.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.450+0000 7f5ca0e91700 1 --2- 172.21.15.43:0/3588375314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c9c06ccc0 0x7f5c9c0acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:32.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.450+0000 7f5c9bfff700 1 -- 172.21.15.43:0/3588375314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c9c06ccc0 msgr2=0x7f5c9c0acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:32.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.450+0000 7f5c9bfff700 1 --2- 172.21.15.43:0/3588375314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c9c06ccc0 0x7f5c9c0acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.450+0000 7f5c9bfff700 1 -- 172.21.15.43:0/3588375314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c9c0af060 msgr2=0x7f5c9c1538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:32.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.450+0000 7f5c9bfff700 1 --2- 172.21.15.43:0/3588375314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c9c0af060 0x7f5c9c1538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.450+0000 7f5c9bfff700 1 -- 172.21.15.43:0/3588375314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c94009a70 con 0x7f5c9c0aec90 2023-06-28T20:07:32.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.450+0000 7f5c9bfff700 1 --2- 172.21.15.43:0/3588375314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c9c0aec90 0x7f5c9c0ad420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800c550 tx=0x7f5c8800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:32.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.450+0000 7f5c997fa700 1 -- 172.21.15.43:0/3588375314 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c88004c80 con 0x7f5c9c0aec90 2023-06-28T20:07:32.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.450+0000 7f5c997fa700 1 -- 172.21.15.43:0/3588375314 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c88004e00 con 0x7f5c9c0aec90 2023-06-28T20:07:32.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.450+0000 7f5c997fa700 1 -- 172.21.15.43:0/3588375314 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c88007310 con 0x7f5c9c0aec90 2023-06-28T20:07:32.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.450+0000 7f5ca30f5700 1 -- 172.21.15.43:0/3588375314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c9c158760 con 0x7f5c9c0aec90 2023-06-28T20:07:32.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.450+0000 7f5ca30f5700 1 -- 172.21.15.43:0/3588375314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9c15f9e0 con 0x7f5c9c0aec90 2023-06-28T20:07:32.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.451+0000 7f5ca30f5700 1 -- 172.21.15.43:0/3588375314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c9c158c80 con 0x7f5c9c0aec90 2023-06-28T20:07:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.452+0000 7f5c997fa700 1 -- 172.21.15.43:0/3588375314 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5c88007470 con 0x7f5c9c0aec90 2023-06-28T20:07:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.452+0000 7f5c997fa700 1 --2- 172.21.15.43:0/3588375314 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5c8006ace0 0x7f5c8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.452+0000 7f5ca0e91700 1 --2- 172.21.15.43:0/3588375314 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5c8006ace0 0x7f5c8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.452+0000 7f5c997fa700 1 -- 172.21.15.43:0/3588375314 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5c880891c0 con 0x7f5c9c0aec90 2023-06-28T20:07:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.453+0000 7f5ca0e91700 1 --2- 172.21.15.43:0/3588375314 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5c8006ace0 0x7f5c8006d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c154480 tx=0x7f5c90008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.457+0000 7f5c997fa700 1 -- 172.21.15.43:0/3588375314 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5c9c158c80 con 0x7f5c9c0aec90 2023-06-28T20:07:32.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.656+0000 7f5ca30f5700 1 -- 172.21.15.43:0/3588375314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5c9c158c80 con 0x7f5c9c0aec90 2023-06-28T20:07:32.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.657+0000 7f5c997fa700 1 -- 172.21.15.43:0/3588375314 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5c9c158c80 con 0x7f5c9c0aec90 2023-06-28T20:07:32.661 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-06-28T20:07:32.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.659+0000 7f5ca30f5700 1 -- 172.21.15.43:0/3588375314 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5c8006ace0 msgr2=0x7f5c8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:32.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.659+0000 7f5ca30f5700 1 --2- 172.21.15.43:0/3588375314 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5c8006ace0 0x7f5c8006d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c154480 tx=0x7f5c90008040 comp rx=0 tx=0).stop 2023-06-28T20:07:32.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.659+0000 7f5ca30f5700 1 -- 172.21.15.43:0/3588375314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c9c0aec90 msgr2=0x7f5c9c0ad420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:32.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.659+0000 7f5ca30f5700 1 --2- 172.21.15.43:0/3588375314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c9c0aec90 0x7f5c9c0ad420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800c550 tx=0x7f5c8800c910 comp rx=0 tx=0).stop 2023-06-28T20:07:32.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.659+0000 7f5ca30f5700 1 -- 172.21.15.43:0/3588375314 shutdown_connections 2023-06-28T20:07:32.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.659+0000 7f5ca30f5700 1 --2- 172.21.15.43:0/3588375314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c9c0af060 0x7f5c9c1538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.659+0000 7f5ca30f5700 1 --2- 172.21.15.43:0/3588375314 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5c8006ace0 0x7f5c8006d190 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.659+0000 7f5ca30f5700 1 --2- 172.21.15.43:0/3588375314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c9c06ccc0 0x7f5c9c0acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.659+0000 7f5ca30f5700 1 --2- 172.21.15.43:0/3588375314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c9c0aec90 0x7f5c9c0ad420 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.659+0000 7f5ca30f5700 1 -- 172.21.15.43:0/3588375314 >> 172.21.15.43:0/3588375314 conn(0x7f5c9c06b000 msgr2=0x7f5c9c0709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:32.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.659+0000 7f5ca30f5700 1 -- 172.21.15.43:0/3588375314 shutdown_connections 2023-06-28T20:07:32.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.659+0000 7f5ca30f5700 1 -- 172.21.15.43:0/3588375314 wait complete. 2023-06-28T20:07:32.674 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-28T20:07:32.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.799+0000 7f3493671700 1 Processor -- start 2023-06-28T20:07:32.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.799+0000 7f3493671700 1 -- start start 2023-06-28T20:07:32.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.800+0000 7f3493671700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f348c0aec90 0x7f348c160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:32.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.800+0000 7f3493671700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f348c0af060 0x7f348c070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:32.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.800+0000 7f3493671700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f348c160250 con 0x7f348c06ccc0 2023-06-28T20:07:32.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.800+0000 7f3493671700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348c15cb30 con 0x7f348c0af060 2023-06-28T20:07:32.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.800+0000 7f3493671700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348c071410 con 0x7f348c0aec90 2023-06-28T20:07:32.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.800+0000 7f3490c0c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f348c0af060 0x7f348c070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:32.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.800+0000 7f3491c0e700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f348c06ccc0 0x7f348c0b1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42668/0 (socket says 172.21.15.43:42668) 2023-06-28T20:07:32.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.800+0000 7f3491c0e700 1 -- 172.21.15.43:0/2312730692 learned_addr learned my addr 172.21.15.43:0/2312730692 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:32.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.801+0000 7f349140d700 1 --2- 172.21.15.43:0/2312730692 >> v2:172.21.15.178:3300/0 conn(0x7f348c0aec90 0x7f348c160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:32.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.801+0000 7f3490c0c700 1 -- 172.21.15.43:0/2312730692 >> v2:172.21.15.178:3300/0 conn(0x7f348c0aec90 msgr2=0x7f348c160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:32.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.801+0000 7f3490c0c700 1 --2- 172.21.15.43:0/2312730692 >> v2:172.21.15.178:3300/0 conn(0x7f348c0aec90 0x7f348c160cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.801+0000 7f3490c0c700 1 -- 172.21.15.43:0/2312730692 >> v1:172.21.15.43:6789/0 conn(0x7f348c06ccc0 legacy=0x7f348c0b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:32.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.801+0000 7f3483fff700 1 -- 172.21.15.43:0/2312730692 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3638472564 0 0) 0x7f348c160250 con 0x7f348c06ccc0 2023-06-28T20:07:32.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.801+0000 7f3490c0c700 1 -- 172.21.15.43:0/2312730692 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c1604f0 con 0x7f348c0af060 2023-06-28T20:07:32.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.801+0000 7f3490c0c700 1 --2- 172.21.15.43:0/2312730692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f348c0af060 0x7f348c070bc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f348c06fce0 tx=0x7f347c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e73d01410f50451 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:32.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.801+0000 7f3483fff700 1 -- 172.21.15.43:0/2312730692 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c011830 con 0x7f348c0af060 2023-06-28T20:07:32.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.801+0000 7f3483fff700 1 -- 172.21.15.43:0/2312730692 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347c011990 con 0x7f348c0af060 2023-06-28T20:07:32.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.801+0000 7f3483fff700 1 -- 172.21.15.43:0/2312730692 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c011d20 con 0x7f348c0af060 2023-06-28T20:07:32.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.802+0000 7f3493671700 1 -- 172.21.15.43:0/2312730692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f348c0af060 msgr2=0x7f348c070bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:32.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.802+0000 7f3493671700 1 --2- 172.21.15.43:0/2312730692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f348c0af060 0x7f348c070bc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f348c06fce0 tx=0x7f347c009b20 comp rx=0 tx=0).stop 2023-06-28T20:07:32.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.802+0000 7f3493671700 1 -- 172.21.15.43:0/2312730692 shutdown_connections 2023-06-28T20:07:32.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.802+0000 7f3493671700 1 --2- 172.21.15.43:0/2312730692 >> v2:172.21.15.178:3300/0 conn(0x7f348c0aec90 0x7f348c160cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.802+0000 7f3493671700 1 --2- 172.21.15.43:0/2312730692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f348c0af060 0x7f348c070bc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.802+0000 7f3493671700 1 -- 172.21.15.43:0/2312730692 >> 172.21.15.43:0/2312730692 conn(0x7f348c06b000 msgr2=0x7f348c06b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:32.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.802+0000 7f3493671700 1 -- 172.21.15.43:0/2312730692 shutdown_connections 2023-06-28T20:07:32.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.802+0000 7f3493671700 1 -- 172.21.15.43:0/2312730692 wait complete. 2023-06-28T20:07:32.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.803+0000 7f3493671700 1 Processor -- start 2023-06-28T20:07:32.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.803+0000 7f3493671700 1 -- start start 2023-06-28T20:07:32.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.804+0000 7f3493671700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f348c06ccc0 0x7f348c15a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:32.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.804+0000 7f3493671700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f348c0aec90 0x7f348c0c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:32.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.804+0000 7f3493671700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f348c0af060 0x7f348c0b4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:32.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.804+0000 7f3493671700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c0b4110 con 0x7f348c06ccc0 2023-06-28T20:07:32.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.804+0000 7f3493671700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c0c76f0 con 0x7f348c0af060 2023-06-28T20:07:32.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.804+0000 7f3493671700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348c0c9a30 con 0x7f348c0aec90 2023-06-28T20:07:32.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.804+0000 7f349140d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f348c06ccc0 0x7f348c15a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:32.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.804+0000 7f349140d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f348c06ccc0 0x7f348c15a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53484/0 (socket says 172.21.15.43:53484) 2023-06-28T20:07:32.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.804+0000 7f349140d700 1 -- 172.21.15.43:0/885163103 learned_addr learned my addr 172.21.15.43:0/885163103 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:32.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.804+0000 7f3491c0e700 1 --2- 172.21.15.43:0/885163103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f348c0af060 0x7f348c0b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:32.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.804+0000 7f3490c0c700 1 --2- 172.21.15.43:0/885163103 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f348c0aec90 0x7f348c0c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:32.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.804+0000 7f349140d700 1 -- 172.21.15.43:0/885163103 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f348c0aec90 msgr2=0x7f348c0c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:32.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.804+0000 7f349140d700 1 --2- 172.21.15.43:0/885163103 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f348c0aec90 0x7f348c0c94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.804+0000 7f349140d700 1 -- 172.21.15.43:0/885163103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f348c0af060 msgr2=0x7f348c0b4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:32.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.804+0000 7f349140d700 1 --2- 172.21.15.43:0/885163103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f348c0af060 0x7f348c0b4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:32.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.804+0000 7f349140d700 1 -- 172.21.15.43:0/885163103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347c0094a0 con 0x7f348c06ccc0 2023-06-28T20:07:32.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.804+0000 7f349140d700 1 --2- 172.21.15.43:0/885163103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f348c06ccc0 0x7f348c15a250 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f348800e810 tx=0x7f348800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:32.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.804+0000 7f3481ffb700 1 -- 172.21.15.43:0/885163103 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488013610 con 0x7f348c06ccc0 2023-06-28T20:07:32.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.804+0000 7f3481ffb700 1 -- 172.21.15.43:0/885163103 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3488013770 con 0x7f348c06ccc0 2023-06-28T20:07:32.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.804+0000 7f3493671700 1 -- 172.21.15.43:0/885163103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f348c0b3170 con 0x7f348c06ccc0 2023-06-28T20:07:32.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.804+0000 7f3481ffb700 1 -- 172.21.15.43:0/885163103 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3488013b20 con 0x7f348c06ccc0 2023-06-28T20:07:32.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.805+0000 7f3493671700 1 -- 172.21.15.43:0/885163103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348c0c70a0 con 0x7f348c06ccc0 2023-06-28T20:07:32.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.806+0000 7f3493671700 1 -- 172.21.15.43:0/885163103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f348c0728b0 con 0x7f348c06ccc0 2023-06-28T20:07:32.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.806+0000 7f3481ffb700 1 -- 172.21.15.43:0/885163103 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3488013c80 con 0x7f348c06ccc0 2023-06-28T20:07:32.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.806+0000 7f3481ffb700 1 --2- 172.21.15.43:0/885163103 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f347006ace0 0x7f347006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:32.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.806+0000 7f3481ffb700 1 -- 172.21.15.43:0/885163103 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f34880891b0 con 0x7f348c06ccc0 2023-06-28T20:07:32.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.806+0000 7f3490c0c700 1 --2- 172.21.15.43:0/885163103 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f347006ace0 0x7f347006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:32.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.807+0000 7f3490c0c700 1 --2- 172.21.15.43:0/885163103 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f347006ace0 0x7f347006d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f348c0c3170 tx=0x7f347c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:32.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:32.811+0000 7f3481ffb700 1 -- 172.21.15.43:0/885163103 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3488054270 con 0x7f348c06ccc0 2023-06-28T20:07:33.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.008+0000 7f3493671700 1 -- 172.21.15.43:0/885163103 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f348c06d9c0 con 0x7f348c06ccc0 2023-06-28T20:07:33.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.009+0000 7f3481ffb700 1 -- 172.21.15.43:0/885163103 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3488057890 con 0x7f348c06ccc0 2023-06-28T20:07:33.012 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-06-28T20:07:33.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.011+0000 7f3493671700 1 -- 172.21.15.43:0/885163103 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f347006ace0 msgr2=0x7f347006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:33.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.011+0000 7f3493671700 1 --2- 172.21.15.43:0/885163103 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f347006ace0 0x7f347006d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f348c0c3170 tx=0x7f347c008030 comp rx=0 tx=0).stop 2023-06-28T20:07:33.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.011+0000 7f3493671700 1 -- 172.21.15.43:0/885163103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f348c06ccc0 msgr2=0x7f348c15a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:33.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.011+0000 7f3493671700 1 --2- 172.21.15.43:0/885163103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f348c06ccc0 0x7f348c15a250 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f348800e810 tx=0x7f348800ebd0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.011+0000 7f3493671700 1 -- 172.21.15.43:0/885163103 shutdown_connections 2023-06-28T20:07:33.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.011+0000 7f3493671700 1 --2- 172.21.15.43:0/885163103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f348c0af060 0x7f348c0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.011+0000 7f3493671700 1 --2- 172.21.15.43:0/885163103 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f348c06ccc0 0x7f348c15a250 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.011+0000 7f3493671700 1 --2- 172.21.15.43:0/885163103 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f347006ace0 0x7f347006d190 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.011+0000 7f3493671700 1 --2- 172.21.15.43:0/885163103 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f348c0aec90 0x7f348c0c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.011+0000 7f3493671700 1 -- 172.21.15.43:0/885163103 >> 172.21.15.43:0/885163103 conn(0x7f348c06b000 msgr2=0x7f348c0cf7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:33.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.011+0000 7f3493671700 1 -- 172.21.15.43:0/885163103 shutdown_connections 2023-06-28T20:07:33.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.011+0000 7f3493671700 1 -- 172.21.15.43:0/885163103 wait complete. 2023-06-28T20:07:33.027 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-28T20:07:33.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.150+0000 7faa72b2e700 1 Processor -- start 2023-06-28T20:07:33.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.150+0000 7faa72b2e700 1 -- start start 2023-06-28T20:07:33.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.151+0000 7faa72b2e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faa6c159820 0x7faa6c15a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:33.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.151+0000 7faa72b2e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa6c06ee20 0x7faa6c0cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:33.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.151+0000 7faa72b2e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6c0ce0a0 con 0x7faa6c0cc600 2023-06-28T20:07:33.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.151+0000 7faa72b2e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa6c0ce380 con 0x7faa6c159820 2023-06-28T20:07:33.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.151+0000 7faa72b2e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c159c90 con 0x7faa6c06ee20 2023-06-28T20:07:33.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.151+0000 7faa6bfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faa6c159820 0x7faa6c15a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:33.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.152+0000 7faa6bfff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faa6c159820 0x7faa6c15a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53492/0 (socket says 172.21.15.43:53492) 2023-06-28T20:07:33.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.152+0000 7faa6bfff700 1 -- 172.21.15.43:0/2811073380 learned_addr learned my addr 172.21.15.43:0/2811073380 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:33.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.152+0000 7faa710cb700 1 --2- 172.21.15.43:0/2811073380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa6c06ee20 0x7faa6c0cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:33.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.152+0000 7faa6bfff700 1 -- 172.21.15.43:0/2811073380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa6c06ee20 msgr2=0x7faa6c0cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:33.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.152+0000 7faa6bfff700 1 --2- 172.21.15.43:0/2811073380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa6c06ee20 0x7faa6c0cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.152+0000 7faa6bfff700 1 -- 172.21.15.43:0/2811073380 >> v1:172.21.15.178:6789/0 conn(0x7faa6c0cc600 legacy=0x7faa6c0cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:33.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.152+0000 7faa6bfff700 1 -- 172.21.15.43:0/2811073380 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6c159f70 con 0x7faa6c159820 2023-06-28T20:07:33.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.152+0000 7faa6bfff700 1 --2- 172.21.15.43:0/2811073380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c159820 0x7faa6c15a490 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7faa5800bb90 tx=0x7faa5800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70ad75cd5594ec5 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:33.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.152+0000 7faa6b7fe700 1 -- 172.21.15.43:0/2811073380 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5800dd50 con 0x7faa6c159820 2023-06-28T20:07:33.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.152+0000 7faa6b7fe700 1 -- 172.21.15.43:0/2811073380 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa58010470 con 0x7faa6c159820 2023-06-28T20:07:33.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.152+0000 7faa6b7fe700 1 -- 172.21.15.43:0/2811073380 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa58010800 con 0x7faa6c159820 2023-06-28T20:07:33.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.153+0000 7faa72b2e700 1 -- 172.21.15.43:0/2811073380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c159820 msgr2=0x7faa6c15a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:33.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.153+0000 7faa72b2e700 1 --2- 172.21.15.43:0/2811073380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c159820 0x7faa6c15a490 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7faa5800bb90 tx=0x7faa5800bea0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.153+0000 7faa72b2e700 1 -- 172.21.15.43:0/2811073380 shutdown_connections 2023-06-28T20:07:33.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.153+0000 7faa72b2e700 1 --2- 172.21.15.43:0/2811073380 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa6c06ee20 0x7faa6c0cd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.153+0000 7faa72b2e700 1 --2- 172.21.15.43:0/2811073380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c159820 0x7faa6c15a490 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.153+0000 7faa72b2e700 1 -- 172.21.15.43:0/2811073380 >> 172.21.15.43:0/2811073380 conn(0x7faa6c0b1720 msgr2=0x7faa6c0b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:33.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.153+0000 7faa72b2e700 1 -- 172.21.15.43:0/2811073380 shutdown_connections 2023-06-28T20:07:33.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.153+0000 7faa72b2e700 1 -- 172.21.15.43:0/2811073380 wait complete. 2023-06-28T20:07:33.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.154+0000 7faa72b2e700 1 Processor -- start 2023-06-28T20:07:33.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.154+0000 7faa72b2e700 1 -- start start 2023-06-28T20:07:33.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.155+0000 7faa72b2e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c06ee20 0x7faa6c14ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:33.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.155+0000 7faa72b2e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa6c0cc600 0x7faa6c151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:33.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.155+0000 7faa72b2e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa6c159820 0x7faa6c151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:33.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.155+0000 7faa72b2e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c14f2b0 con 0x7faa6c06ee20 2023-06-28T20:07:33.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.155+0000 7faa72b2e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c0c4930 con 0x7faa6c159820 2023-06-28T20:07:33.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.155+0000 7faa72b2e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c0c4c10 con 0x7faa6c0cc600 2023-06-28T20:07:33.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.155+0000 7faa708ca700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c06ee20 0x7faa6c14ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:33.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.155+0000 7faa708ca700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c06ee20 0x7faa6c14ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53498/0 (socket says 172.21.15.43:53498) 2023-06-28T20:07:33.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.155+0000 7faa708ca700 1 -- 172.21.15.43:0/1448909886 learned_addr learned my addr 172.21.15.43:0/1448909886 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:33.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.155+0000 7faa6bfff700 1 --2- 172.21.15.43:0/1448909886 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa6c0cc600 0x7faa6c151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:33.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.155+0000 7faa710cb700 1 --2- 172.21.15.43:0/1448909886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa6c159820 0x7faa6c151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:33.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.155+0000 7faa708ca700 1 -- 172.21.15.43:0/1448909886 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa6c0cc600 msgr2=0x7faa6c151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:33.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.155+0000 7faa708ca700 1 --2- 172.21.15.43:0/1448909886 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa6c0cc600 0x7faa6c151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.155+0000 7faa708ca700 1 -- 172.21.15.43:0/1448909886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa6c159820 msgr2=0x7faa6c151840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:33.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.155+0000 7faa708ca700 1 --2- 172.21.15.43:0/1448909886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa6c159820 0x7faa6c151840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.155+0000 7faa708ca700 1 -- 172.21.15.43:0/1448909886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5800b7f0 con 0x7faa6c06ee20 2023-06-28T20:07:33.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.155+0000 7faa710cb700 1 --2- 172.21.15.43:0/1448909886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa6c159820 0x7faa6c151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:33.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.156+0000 7faa708ca700 1 --2- 172.21.15.43:0/1448909886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c06ee20 0x7faa6c14ed70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faa6000c4e0 tx=0x7faa6000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:33.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.156+0000 7faa697fa700 1 -- 172.21.15.43:0/1448909886 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa600052d0 con 0x7faa6c06ee20 2023-06-28T20:07:33.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.156+0000 7faa697fa700 1 -- 172.21.15.43:0/1448909886 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa60005430 con 0x7faa6c06ee20 2023-06-28T20:07:33.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.156+0000 7faa72b2e700 1 -- 172.21.15.43:0/1448909886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa6c0c4ef0 con 0x7faa6c06ee20 2023-06-28T20:07:33.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.156+0000 7faa697fa700 1 -- 172.21.15.43:0/1448909886 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa600038c0 con 0x7faa6c06ee20 2023-06-28T20:07:33.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.156+0000 7faa72b2e700 1 -- 172.21.15.43:0/1448909886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa6c0c54b0 con 0x7faa6c06ee20 2023-06-28T20:07:33.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.157+0000 7faa72b2e700 1 -- 172.21.15.43:0/1448909886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa6c0728b0 con 0x7faa6c06ee20 2023-06-28T20:07:33.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.157+0000 7faa697fa700 1 -- 172.21.15.43:0/1448909886 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faa60003a20 con 0x7faa6c06ee20 2023-06-28T20:07:33.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.157+0000 7faa697fa700 1 --2- 172.21.15.43:0/1448909886 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7faa5006ace0 0x7faa5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:33.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.158+0000 7faa697fa700 1 -- 172.21.15.43:0/1448909886 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faa600889a0 con 0x7faa6c06ee20 2023-06-28T20:07:33.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.158+0000 7faa6bfff700 1 --2- 172.21.15.43:0/1448909886 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7faa5006ace0 0x7faa5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:33.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.158+0000 7faa6bfff700 1 --2- 172.21.15.43:0/1448909886 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7faa5006ace0 0x7faa5006d190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faa5800bee0 tx=0x7faa58013000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:33.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.162+0000 7faa697fa700 1 -- 172.21.15.43:0/1448909886 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7faa60053a50 con 0x7faa6c06ee20 2023-06-28T20:07:33.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.406+0000 7faa72b2e700 1 -- 172.21.15.43:0/1448909886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa6c06ade0 con 0x7faa6c06ee20 2023-06-28T20:07:33.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.407+0000 7faa697fa700 1 -- 172.21.15.43:0/1448909886 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4665 (secure 0 0 0) 0x7faa60057070 con 0x7faa6c06ee20 2023-06-28T20:07:33.411 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:33.411 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4033115951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4033115951},{"type":"v1","addr":"172.21.15.178:6835","nonce":4033115951}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/65569869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":65569869},{"type":"v1","addr":"172.21.15.43:6837","nonce":65569869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/1016213057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016213057},{"type":"v1","addr":"172.21.15.178:6834","nonce":1016213057}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-28T20:07:20.788417+0000","modified":"2023-06-28T20:07:24.388714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/3514544484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3514544484},{"type":"v1","addr":"172.21.15.43:6835","nonce":3514544484}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-28T20:07:33.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.409+0000 7faa72b2e700 1 -- 172.21.15.43:0/1448909886 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7faa5006ace0 msgr2=0x7faa5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:33.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.409+0000 7faa72b2e700 1 --2- 172.21.15.43:0/1448909886 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7faa5006ace0 0x7faa5006d190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faa5800bee0 tx=0x7faa58013000 comp rx=0 tx=0).stop 2023-06-28T20:07:33.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.409+0000 7faa72b2e700 1 -- 172.21.15.43:0/1448909886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c06ee20 msgr2=0x7faa6c14ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:33.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.409+0000 7faa72b2e700 1 --2- 172.21.15.43:0/1448909886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c06ee20 0x7faa6c14ed70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7faa6000c4e0 tx=0x7faa6000c8a0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.409+0000 7faa72b2e700 1 -- 172.21.15.43:0/1448909886 shutdown_connections 2023-06-28T20:07:33.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.409+0000 7faa72b2e700 1 --2- 172.21.15.43:0/1448909886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa6c159820 0x7faa6c151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.409+0000 7faa72b2e700 1 --2- 172.21.15.43:0/1448909886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c06ee20 0x7faa6c14ed70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.409+0000 7faa72b2e700 1 --2- 172.21.15.43:0/1448909886 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7faa5006ace0 0x7faa5006d190 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.409+0000 7faa72b2e700 1 --2- 172.21.15.43:0/1448909886 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faa6c0cc600 0x7faa6c151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.409+0000 7faa72b2e700 1 -- 172.21.15.43:0/1448909886 >> 172.21.15.43:0/1448909886 conn(0x7faa6c0b1720 msgr2=0x7faa6c0d04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.409+0000 7faa72b2e700 1 -- 172.21.15.43:0/1448909886 shutdown_connections 2023-06-28T20:07:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.409+0000 7faa72b2e700 1 -- 172.21.15.43:0/1448909886 wait complete. 2023-06-28T20:07:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 7 2023-06-28T20:07:33.424 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:07:33.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.546+0000 7f0beab56700 1 Processor -- start 2023-06-28T20:07:33.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.546+0000 7f0beab56700 1 -- start start 2023-06-28T20:07:33.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.546+0000 7f0beab56700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0be40d10f0 0x7f0be40ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:33.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.546+0000 7f0beab56700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0be40cbab0 0x7f0be40b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:33.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.546+0000 7f0beab56700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0be415ab50 con 0x7f0be40cedf0 2023-06-28T20:07:33.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.546+0000 7f0beab56700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be40cbf30 con 0x7f0be40d10f0 2023-06-28T20:07:33.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.546+0000 7f0beab56700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be40cf260 con 0x7f0be40cbab0 2023-06-28T20:07:33.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.547+0000 7f0be3fff700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f0be40cedf0 0x7f0be40cb3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:41818/0 (socket says 172.21.15.43:41818) 2023-06-28T20:07:33.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.547+0000 7f0be90f3700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0be40cbab0 0x7f0be40b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:33.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.547+0000 7f0be88f2700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0be40d10f0 0x7f0be40ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:33.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.547+0000 7f0be3fff700 1 -- 172.21.15.43:0/1842577209 learned_addr learned my addr 172.21.15.43:0/1842577209 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:33.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.547+0000 7f0be88f2700 1 -- 172.21.15.43:0/1842577209 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0be40cbab0 msgr2=0x7f0be40b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:33.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.547+0000 7f0be88f2700 1 --2- 172.21.15.43:0/1842577209 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0be40cbab0 0x7f0be40b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.547+0000 7f0be88f2700 1 -- 172.21.15.43:0/1842577209 >> v1:172.21.15.178:6789/0 conn(0x7f0be40cedf0 legacy=0x7f0be40cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:33.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.547+0000 7f0be37fe700 1 -- 172.21.15.43:0/1842577209 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1315394018 0 0) 0x7f0be415ab50 con 0x7f0be40cedf0 2023-06-28T20:07:33.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.547+0000 7f0be88f2700 1 -- 172.21.15.43:0/1842577209 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be40cd1b0 con 0x7f0be40d10f0 2023-06-28T20:07:33.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.547+0000 7f0be90f3700 1 --2- 172.21.15.43:0/1842577209 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0be40cbab0 0x7f0be40b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.547+0000 7f0be88f2700 1 --2- 172.21.15.43:0/1842577209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0be40d10f0 0x7f0be40ce8b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8009850 tx=0x7f0bd8009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68133177e7e139e server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.548+0000 7f0be37fe700 1 -- 172.21.15.43:0/1842577209 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8005e30 con 0x7f0be40d10f0 2023-06-28T20:07:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.548+0000 7f0be37fe700 1 -- 172.21.15.43:0/1842577209 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd8006030 con 0x7f0be40d10f0 2023-06-28T20:07:33.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.548+0000 7f0be37fe700 1 -- 172.21.15.43:0/1842577209 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd80063c0 con 0x7f0be40d10f0 2023-06-28T20:07:33.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.548+0000 7f0beab56700 1 -- 172.21.15.43:0/1842577209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0be40d10f0 msgr2=0x7f0be40ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:33.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.548+0000 7f0beab56700 1 --2- 172.21.15.43:0/1842577209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0be40d10f0 0x7f0be40ce8b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8009850 tx=0x7f0bd8009b60 comp rx=0 tx=0).stop 2023-06-28T20:07:33.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.548+0000 7f0beab56700 1 -- 172.21.15.43:0/1842577209 shutdown_connections 2023-06-28T20:07:33.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.548+0000 7f0beab56700 1 --2- 172.21.15.43:0/1842577209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0be40d10f0 0x7f0be40ce8b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.548+0000 7f0beab56700 1 --2- 172.21.15.43:0/1842577209 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0be40cbab0 0x7f0be40b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.548+0000 7f0beab56700 1 -- 172.21.15.43:0/1842577209 >> 172.21.15.43:0/1842577209 conn(0x7f0be40664b0 msgr2=0x7f0be409a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.548+0000 7f0beab56700 1 -- 172.21.15.43:0/1842577209 shutdown_connections 2023-06-28T20:07:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.548+0000 7f0beab56700 1 -- 172.21.15.43:0/1842577209 wait complete. 2023-06-28T20:07:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.549+0000 7f0beab56700 1 Processor -- start 2023-06-28T20:07:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.549+0000 7f0beab56700 1 -- start start 2023-06-28T20:07:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.550+0000 7f0beab56700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0be40cbab0 0x7f0be41f5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.550+0000 7f0beab56700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0be40cedf0 0x7f0be41f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:33.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.550+0000 7f0beab56700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0be40d10f0 0x7f0be40cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:33.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.550+0000 7f0beab56700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be406c9d0 con 0x7f0be40cedf0 2023-06-28T20:07:33.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.550+0000 7f0beab56700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be406be50 con 0x7f0be40d10f0 2023-06-28T20:07:33.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.550+0000 7f0beab56700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be406c130 con 0x7f0be40cbab0 2023-06-28T20:07:33.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.550+0000 7f0be3fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0be40cedf0 0x7f0be41f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:33.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.550+0000 7f0be3fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0be40cedf0 0x7f0be41f5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53514/0 (socket says 172.21.15.43:53514) 2023-06-28T20:07:33.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.550+0000 7f0be3fff700 1 -- 172.21.15.43:0/353173944 learned_addr learned my addr 172.21.15.43:0/353173944 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:33.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.550+0000 7f0be90f3700 1 --2- 172.21.15.43:0/353173944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0be40d10f0 0x7f0be40cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:33.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.550+0000 7f0be88f2700 1 --2- 172.21.15.43:0/353173944 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0be40cbab0 0x7f0be41f5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:33.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.551+0000 7f0be3fff700 1 -- 172.21.15.43:0/353173944 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0be40cbab0 msgr2=0x7f0be41f5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:33.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.551+0000 7f0be3fff700 1 --2- 172.21.15.43:0/353173944 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0be40cbab0 0x7f0be41f5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.551+0000 7f0be3fff700 1 -- 172.21.15.43:0/353173944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0be40d10f0 msgr2=0x7f0be40cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:33.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.551+0000 7f0be3fff700 1 --2- 172.21.15.43:0/353173944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0be40d10f0 0x7f0be40cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.551+0000 7f0be3fff700 1 -- 172.21.15.43:0/353173944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bdc009a70 con 0x7f0be40cedf0 2023-06-28T20:07:33.558 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.551+0000 7f0be3fff700 1 --2- 172.21.15.43:0/353173944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0be40cedf0 0x7f0be41f5b70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00011e0 tx=0x7f0bd000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.551+0000 7f0be17fa700 1 -- 172.21.15.43:0/353173944 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0004c80 con 0x7f0be40cedf0 2023-06-28T20:07:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.551+0000 7f0be17fa700 1 -- 172.21.15.43:0/353173944 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd0004de0 con 0x7f0be40cedf0 2023-06-28T20:07:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.551+0000 7f0beab56700 1 -- 172.21.15.43:0/353173944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd8009430 con 0x7f0be40cedf0 2023-06-28T20:07:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.551+0000 7f0be17fa700 1 -- 172.21.15.43:0/353173944 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0007310 con 0x7f0be40cedf0 2023-06-28T20:07:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.551+0000 7f0beab56700 1 -- 172.21.15.43:0/353173944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be406c6f0 con 0x7f0be40cedf0 2023-06-28T20:07:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.552+0000 7f0beab56700 1 -- 172.21.15.43:0/353173944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be40cfda0 con 0x7f0be40cedf0 2023-06-28T20:07:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.553+0000 7f0be17fa700 1 -- 172.21.15.43:0/353173944 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0bd0007470 con 0x7f0be40cedf0 2023-06-28T20:07:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.553+0000 7f0be17fa700 1 --2- 172.21.15.43:0/353173944 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0bc806ace0 0x7f0bc806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.553+0000 7f0be17fa700 1 -- 172.21.15.43:0/353173944 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0bd00890e0 con 0x7f0be40cedf0 2023-06-28T20:07:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.553+0000 7f0be88f2700 1 --2- 172.21.15.43:0/353173944 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0bc806ace0 0x7f0bc806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:33.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.553+0000 7f0be88f2700 1 --2- 172.21.15.43:0/353173944 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0bc806ace0 0x7f0bc806d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8013040 tx=0x7f0bd8008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:33.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.557+0000 7f0be17fa700 1 -- 172.21.15.43:0/353173944 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0bd0054190 con 0x7f0be40cedf0 2023-06-28T20:07:33.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.754+0000 7f0beab56700 1 -- 172.21.15.43:0/353173944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0be40b1c80 con 0x7f0be40cedf0 2023-06-28T20:07:33.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.755+0000 7f0be17fa700 1 -- 172.21.15.43:0/353173944 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18221 (secure 0 0 0) 0x7f0bd00577b0 con 0x7f0be40cedf0 2023-06-28T20:07:33.759 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:33.759 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:07:22.377248+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:07:18.624759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-28T20:07:19.760017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:07:33.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.757+0000 7f0beab56700 1 -- 172.21.15.43:0/353173944 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0bc806ace0 msgr2=0x7f0bc806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:33.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.757+0000 7f0beab56700 1 --2- 172.21.15.43:0/353173944 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0bc806ace0 0x7f0bc806d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0bd8013040 tx=0x7f0bd8008070 comp rx=0 tx=0).stop 2023-06-28T20:07:33.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.757+0000 7f0beab56700 1 -- 172.21.15.43:0/353173944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0be40cedf0 msgr2=0x7f0be41f5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:33.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.757+0000 7f0beab56700 1 --2- 172.21.15.43:0/353173944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0be40cedf0 0x7f0be41f5b70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00011e0 tx=0x7f0bd000ca30 comp rx=0 tx=0).stop 2023-06-28T20:07:33.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.757+0000 7f0beab56700 1 -- 172.21.15.43:0/353173944 shutdown_connections 2023-06-28T20:07:33.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.757+0000 7f0beab56700 1 --2- 172.21.15.43:0/353173944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0be40d10f0 0x7f0be40cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.757+0000 7f0beab56700 1 --2- 172.21.15.43:0/353173944 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0bc806ace0 0x7f0bc806d190 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.757+0000 7f0beab56700 1 --2- 172.21.15.43:0/353173944 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0be40cbab0 0x7f0be41f5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.757+0000 7f0beab56700 1 --2- 172.21.15.43:0/353173944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0be40cedf0 0x7f0be41f5b70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.757+0000 7f0beab56700 1 -- 172.21.15.43:0/353173944 >> 172.21.15.43:0/353173944 conn(0x7f0be40664b0 msgr2=0x7f0be415a250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.757+0000 7f0beab56700 1 -- 172.21.15.43:0/353173944 shutdown_connections 2023-06-28T20:07:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.758+0000 7f0beab56700 1 -- 172.21.15.43:0/353173944 wait complete. 2023-06-28T20:07:33.774 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-28T20:07:13.580969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-28T20:07:14.038358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-28T20:07:18.624759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-28T20:07:19.760017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-28T20:07:33.775 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-28T20:07:33.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.900+0000 7f52c70c8700 1 Processor -- start 2023-06-28T20:07:33.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.900+0000 7f52c70c8700 1 -- start start 2023-06-28T20:07:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.901+0000 7f52c70c8700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f52c006c7b0 0x7f52c006ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.901+0000 7f52c70c8700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c00ce370 con 0x7f52c00c9900 2023-06-28T20:07:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.901+0000 7f52c70c8700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c00ce610 con 0x7f52c006a830 2023-06-28T20:07:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.901+0000 7f52c70c8700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c00c9d80 con 0x7f52c006c7b0 2023-06-28T20:07:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.901+0000 7f52bffff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f52c00c9900 0x7f52c006c0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42708/0 (socket says 172.21.15.43:42708) 2023-06-28T20:07:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.901+0000 7f52bffff700 1 -- 172.21.15.43:0/3798237865 learned_addr learned my addr 172.21.15.43:0/3798237865 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.901+0000 7f52c5665700 1 --2- 172.21.15.43:0/3798237865 >> v2:172.21.15.178:3300/0 conn(0x7f52c006c7b0 0x7f52c006ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.901+0000 7f52bf7fe700 1 -- 172.21.15.43:0/3798237865 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635755168 0 0) 0x7f52c00ce370 con 0x7f52c00c9900 2023-06-28T20:07:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.901+0000 7f52bf7fe700 1 -- 172.21.15.43:0/3798237865 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52b00035b0 con 0x7f52c00c9900 2023-06-28T20:07:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.902+0000 7f52bf7fe700 1 -- 172.21.15.43:0/3798237865 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2602378610 0 0) 0x7f52c00ce610 con 0x7f52c006a830 2023-06-28T20:07:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.902+0000 7f52bf7fe700 1 -- 172.21.15.43:0/3798237865 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52c00ce370 con 0x7f52c006a830 2023-06-28T20:07:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.902+0000 7f52c5665700 1 -- 172.21.15.43:0/3798237865 >> v1:172.21.15.178:6789/0 conn(0x7f52c006a830 legacy=0x7f52c0069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.902+0000 7f52c5665700 1 -- 172.21.15.43:0/3798237865 >> v1:172.21.15.43:6789/0 conn(0x7f52c00c9900 legacy=0x7f52c006c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.902+0000 7f52bf7fe700 1 -- 172.21.15.43:0/3798237865 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2558497603 0 0) 0x7f52b00035b0 con 0x7f52c00c9900 2023-06-28T20:07:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.902+0000 7f52c5665700 1 -- 172.21.15.43:0/3798237865 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c00ce8f0 con 0x7f52c006c7b0 2023-06-28T20:07:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.902+0000 7f52c5665700 1 --2- 172.21.15.43:0/3798237865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52c006c7b0 0x7f52c006ac00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f52b800aea0 tx=0x7f52b800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb25cb6170027bd7 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.902+0000 7f52bf7fe700 1 -- 172.21.15.43:0/3798237865 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b800f070 con 0x7f52c006c7b0 2023-06-28T20:07:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.902+0000 7f52bf7fe700 1 -- 172.21.15.43:0/3798237865 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b8003a20 con 0x7f52c006c7b0 2023-06-28T20:07:33.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.902+0000 7f52bf7fe700 1 -- 172.21.15.43:0/3798237865 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ac001670 con 0x7f52c006c7b0 2023-06-28T20:07:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.902+0000 7f52c70c8700 1 -- 172.21.15.43:0/3798237865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52c006c7b0 msgr2=0x7f52c006ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.902+0000 7f52c70c8700 1 --2- 172.21.15.43:0/3798237865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52c006c7b0 0x7f52c006ac00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f52b800aea0 tx=0x7f52b800c330 comp rx=0 tx=0).stop 2023-06-28T20:07:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.903+0000 7f52c70c8700 1 -- 172.21.15.43:0/3798237865 shutdown_connections 2023-06-28T20:07:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.903+0000 7f52c70c8700 1 --2- 172.21.15.43:0/3798237865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52c006c7b0 0x7f52c006ac00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.903+0000 7f52c70c8700 1 -- 172.21.15.43:0/3798237865 >> 172.21.15.43:0/3798237865 conn(0x7f52c00b1a50 msgr2=0x7f52c00cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.903+0000 7f52c70c8700 1 -- 172.21.15.43:0/3798237865 shutdown_connections 2023-06-28T20:07:33.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.903+0000 7f52c70c8700 1 -- 172.21.15.43:0/3798237865 wait complete. 2023-06-28T20:07:33.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.904+0000 7f52c70c8700 1 Processor -- start 2023-06-28T20:07:33.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.904+0000 7f52c70c8700 1 -- start start 2023-06-28T20:07:33.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.905+0000 7f52c70c8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52c006a830 0x7f52c015bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:33.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.905+0000 7f52c70c8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52c006c7b0 0x7f52c00af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:33.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.905+0000 7f52c70c8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52c00c9900 0x7f52c00aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:33.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.905+0000 7f52c70c8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c015c0a0 con 0x7f52c006a830 2023-06-28T20:07:33.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.905+0000 7f52c70c8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c00c6520 con 0x7f52c006c7b0 2023-06-28T20:07:33.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.905+0000 7f52c70c8700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c00c6800 con 0x7f52c00c9900 2023-06-28T20:07:33.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.905+0000 7f52c4e64700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52c006a830 0x7f52c015bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:33.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.905+0000 7f52c4e64700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52c006a830 0x7f52c015bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53526/0 (socket says 172.21.15.43:53526) 2023-06-28T20:07:33.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.905+0000 7f52c4e64700 1 -- 172.21.15.43:0/1657951951 learned_addr learned my addr 172.21.15.43:0/1657951951 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:33.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.905+0000 7f52bffff700 1 --2- 172.21.15.43:0/1657951951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52c006c7b0 0x7f52c00af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:33.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.905+0000 7f52c5665700 1 --2- 172.21.15.43:0/1657951951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52c00c9900 0x7f52c00aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:33.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.905+0000 7f52c4e64700 1 -- 172.21.15.43:0/1657951951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52c00c9900 msgr2=0x7f52c00aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:33.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.905+0000 7f52c4e64700 1 --2- 172.21.15.43:0/1657951951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52c00c9900 0x7f52c00aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.905+0000 7f52c4e64700 1 -- 172.21.15.43:0/1657951951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52c006c7b0 msgr2=0x7f52c00af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:33.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.905+0000 7f52c4e64700 1 --2- 172.21.15.43:0/1657951951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52c006c7b0 0x7f52c00af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:33.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.905+0000 7f52c4e64700 1 -- 172.21.15.43:0/1657951951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52b800ab80 con 0x7f52c006a830 2023-06-28T20:07:33.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.905+0000 7f52c5665700 1 --2- 172.21.15.43:0/1657951951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52c00c9900 0x7f52c00aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:33.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.905+0000 7f52bffff700 1 --2- 172.21.15.43:0/1657951951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52c006c7b0 0x7f52c00af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:33.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.905+0000 7f52c4e64700 1 --2- 172.21.15.43:0/1657951951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52c006a830 0x7f52c015bb60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f52b400c500 tx=0x7f52b400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:33.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.905+0000 7f52bd7fa700 1 -- 172.21.15.43:0/1657951951 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b40052d0 con 0x7f52c006a830 2023-06-28T20:07:33.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.905+0000 7f52bd7fa700 1 -- 172.21.15.43:0/1657951951 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b4005450 con 0x7f52c006a830 2023-06-28T20:07:33.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.905+0000 7f52bd7fa700 1 -- 172.21.15.43:0/1657951951 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b40038c0 con 0x7f52c006a830 2023-06-28T20:07:33.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.906+0000 7f52c70c8700 1 -- 172.21.15.43:0/1657951951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c00c6ae0 con 0x7f52c006a830 2023-06-28T20:07:33.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.906+0000 7f52c70c8700 1 -- 172.21.15.43:0/1657951951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c00c70a0 con 0x7f52c006a830 2023-06-28T20:07:33.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.907+0000 7f52c70c8700 1 -- 172.21.15.43:0/1657951951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c00c72c0 con 0x7f52c006a830 2023-06-28T20:07:33.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.907+0000 7f52bd7fa700 1 -- 172.21.15.43:0/1657951951 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f52b4003a20 con 0x7f52c006a830 2023-06-28T20:07:33.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.907+0000 7f52bd7fa700 1 --2- 172.21.15.43:0/1657951951 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f52a406ace0 0x7f52a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:33.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.908+0000 7f52bd7fa700 1 -- 172.21.15.43:0/1657951951 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f52b4088a20 con 0x7f52c006a830 2023-06-28T20:07:33.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.908+0000 7f52bffff700 1 --2- 172.21.15.43:0/1657951951 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f52a406ace0 0x7f52a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:33.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.908+0000 7f52bffff700 1 --2- 172.21.15.43:0/1657951951 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f52a406ace0 0x7f52a406d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f52ac001eb0 tx=0x7f52ac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:33.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:33.912+0000 7f52bd7fa700 1 -- 172.21.15.43:0/1657951951 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f52b4053a50 con 0x7f52c006a830 2023-06-28T20:07:34.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.112+0000 7f52c70c8700 1 -- 172.21.15.43:0/1657951951 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f52c00728b0 con 0x7f52c006a830 2023-06-28T20:07:34.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.112+0000 7f52bd7fa700 1 -- 172.21.15.43:0/1657951951 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f52b4057070 con 0x7f52c006a830 2023-06-28T20:07:34.115 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-06-28T20:07:34.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.114+0000 7f52c70c8700 1 -- 172.21.15.43:0/1657951951 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f52a406ace0 msgr2=0x7f52a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:34.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.114+0000 7f52c70c8700 1 --2- 172.21.15.43:0/1657951951 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f52a406ace0 0x7f52a406d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f52ac001eb0 tx=0x7f52ac009040 comp rx=0 tx=0).stop 2023-06-28T20:07:34.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.114+0000 7f52c70c8700 1 -- 172.21.15.43:0/1657951951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52c006a830 msgr2=0x7f52c015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:34.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.114+0000 7f52c70c8700 1 --2- 172.21.15.43:0/1657951951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52c006a830 0x7f52c015bb60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f52b400c500 tx=0x7f52b400c8c0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.114+0000 7f52c70c8700 1 -- 172.21.15.43:0/1657951951 shutdown_connections 2023-06-28T20:07:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.114+0000 7f52c70c8700 1 --2- 172.21.15.43:0/1657951951 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f52a406ace0 0x7f52a406d190 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.114+0000 7f52c70c8700 1 --2- 172.21.15.43:0/1657951951 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52c00c9900 0x7f52c00aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.114+0000 7f52c70c8700 1 --2- 172.21.15.43:0/1657951951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52c006a830 0x7f52c015bb60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.114+0000 7f52c70c8700 1 --2- 172.21.15.43:0/1657951951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52c006c7b0 0x7f52c00af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.114+0000 7f52c70c8700 1 -- 172.21.15.43:0/1657951951 >> 172.21.15.43:0/1657951951 conn(0x7f52c00b1a50 msgr2=0x7f52c006c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.114+0000 7f52c70c8700 1 -- 172.21.15.43:0/1657951951 shutdown_connections 2023-06-28T20:07:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.114+0000 7f52c70c8700 1 -- 172.21.15.43:0/1657951951 wait complete. 2023-06-28T20:07:34.129 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-28T20:07:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.254+0000 7f1c95735700 1 Processor -- start 2023-06-28T20:07:34.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.254+0000 7f1c95735700 1 -- start start 2023-06-28T20:07:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.255+0000 7f1c95735700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1c900b20e0 0x7f1c9015d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.255+0000 7f1c95735700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c901596a0 0x7f1c9015d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.255+0000 7f1c95735700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c90070b40 con 0x7f1c901592d0 2023-06-28T20:07:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.255+0000 7f1c95735700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c90070e20 con 0x7f1c900b20e0 2023-06-28T20:07:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.255+0000 7f1c95735700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c90071100 con 0x7f1c901596a0 2023-06-28T20:07:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.256+0000 7f1c8f7fe700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c901596a0 0x7f1c9015d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.256+0000 7f1c8e7fc700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1c900b20e0 0x7f1c9015d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.256+0000 7f1c8f7fe700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c901596a0 0x7f1c9015d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.43:37510/0 (socket says 172.21.15.43:37510) 2023-06-28T20:07:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.256+0000 7f1c8e7fc700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1c900b20e0 0x7f1c9015d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:39600/0 (socket says 172.21.15.43:39600) 2023-06-28T20:07:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.256+0000 7f1c8f7fe700 1 -- 172.21.15.43:0/1292176232 learned_addr learned my addr 172.21.15.43:0/1292176232 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.256+0000 7f1c8effd700 1 --1- 172.21.15.43:0/1292176232 >> v1:172.21.15.178:6789/0 conn(0x7f1c901592d0 0x7f1c900b19d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:41844/0 (socket says 172.21.15.43:41844) 2023-06-28T20:07:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.256+0000 7f1c8f7fe700 1 -- 172.21.15.43:0/1292176232 >> v2:172.21.15.178:3300/0 conn(0x7f1c900b20e0 msgr2=0x7f1c9015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.256+0000 7f1c8f7fe700 1 --2- 172.21.15.43:0/1292176232 >> v2:172.21.15.178:3300/0 conn(0x7f1c900b20e0 0x7f1c9015d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.256+0000 7f1c8f7fe700 1 -- 172.21.15.43:0/1292176232 >> v1:172.21.15.178:6789/0 conn(0x7f1c901592d0 legacy=0x7f1c900b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.256+0000 7f1c8f7fe700 1 -- 172.21.15.43:0/1292176232 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c900713e0 con 0x7f1c901596a0 2023-06-28T20:07:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.256+0000 7f1c8f7fe700 1 --2- 172.21.15.43:0/1292176232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c901596a0 0x7f1c9015d5b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009e50 tx=0x7f1c8800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b15397b108cc2f3f server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:34.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.256+0000 7f1c8dffb700 1 -- 172.21.15.43:0/1292176232 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8800e070 con 0x7f1c901596a0 2023-06-28T20:07:34.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.256+0000 7f1c8dffb700 1 -- 172.21.15.43:0/1292176232 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8800b910 con 0x7f1c901596a0 2023-06-28T20:07:34.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.256+0000 7f1c8dffb700 1 -- 172.21.15.43:0/1292176232 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8800bcc0 con 0x7f1c901596a0 2023-06-28T20:07:34.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.257+0000 7f1c95735700 1 -- 172.21.15.43:0/1292176232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c901596a0 msgr2=0x7f1c9015d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:34.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.257+0000 7f1c95735700 1 --2- 172.21.15.43:0/1292176232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c901596a0 0x7f1c9015d5b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009e50 tx=0x7f1c8800b330 comp rx=0 tx=0).stop 2023-06-28T20:07:34.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.257+0000 7f1c95735700 1 -- 172.21.15.43:0/1292176232 shutdown_connections 2023-06-28T20:07:34.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.257+0000 7f1c95735700 1 --2- 172.21.15.43:0/1292176232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c901596a0 0x7f1c9015d5b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.257+0000 7f1c95735700 1 --2- 172.21.15.43:0/1292176232 >> v2:172.21.15.178:3300/0 conn(0x7f1c900b20e0 0x7f1c9015d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.257+0000 7f1c95735700 1 -- 172.21.15.43:0/1292176232 >> 172.21.15.43:0/1292176232 conn(0x7f1c9006b000 msgr2=0x7f1c9006b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:34.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.257+0000 7f1c95735700 1 -- 172.21.15.43:0/1292176232 shutdown_connections 2023-06-28T20:07:34.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.257+0000 7f1c95735700 1 -- 172.21.15.43:0/1292176232 wait complete. 2023-06-28T20:07:34.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.258+0000 7f1c95735700 1 Processor -- start 2023-06-28T20:07:34.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.258+0000 7f1c95735700 1 -- start start 2023-06-28T20:07:34.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.259+0000 7f1c95735700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c900b20e0 0x7f1c90152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:34.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.259+0000 7f1c95735700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c901592d0 0x7f1c900b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:34.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.259+0000 7f1c95735700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c901596a0 0x7f1c9015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:34.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.259+0000 7f1c95735700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c900c6500 con 0x7f1c900b20e0 2023-06-28T20:07:34.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.259+0000 7f1c95735700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c90159aa0 con 0x7f1c901592d0 2023-06-28T20:07:34.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.259+0000 7f1c95735700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c90159cf0 con 0x7f1c901596a0 2023-06-28T20:07:34.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.259+0000 7f1c8effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c900b20e0 0x7f1c90152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:34.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.259+0000 7f1c8effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c900b20e0 0x7f1c90152c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53540/0 (socket says 172.21.15.43:53540) 2023-06-28T20:07:34.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.259+0000 7f1c8effd700 1 -- 172.21.15.43:0/95850533 learned_addr learned my addr 172.21.15.43:0/95850533 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:34.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.259+0000 7f1c8e7fc700 1 --2- 172.21.15.43:0/95850533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c901592d0 0x7f1c900b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:34.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.259+0000 7f1c8f7fe700 1 --2- 172.21.15.43:0/95850533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c901596a0 0x7f1c9015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:34.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.260+0000 7f1c8effd700 1 -- 172.21.15.43:0/95850533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c901596a0 msgr2=0x7f1c9015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:34.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.260+0000 7f1c8effd700 1 --2- 172.21.15.43:0/95850533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c901596a0 0x7f1c9015ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.260+0000 7f1c8effd700 1 -- 172.21.15.43:0/95850533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c901592d0 msgr2=0x7f1c900b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:34.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.260+0000 7f1c8effd700 1 --2- 172.21.15.43:0/95850533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c901592d0 0x7f1c900b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.260+0000 7f1c8effd700 1 -- 172.21.15.43:0/95850533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c88009b30 con 0x7f1c900b20e0 2023-06-28T20:07:34.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.260+0000 7f1c8e7fc700 1 --2- 172.21.15.43:0/95850533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c901592d0 0x7f1c900b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:34.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.260+0000 7f1c8f7fe700 1 --2- 172.21.15.43:0/95850533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c901596a0 0x7f1c9015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:34.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.260+0000 7f1c8effd700 1 --2- 172.21.15.43:0/95850533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c900b20e0 0x7f1c90152c00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400e730 tx=0x7f1c8400eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:34.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.260+0000 7f1c777fe700 1 -- 172.21.15.43:0/95850533 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c84009070 con 0x7f1c900b20e0 2023-06-28T20:07:34.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.260+0000 7f1c777fe700 1 -- 172.21.15.43:0/95850533 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c840091d0 con 0x7f1c900b20e0 2023-06-28T20:07:34.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.260+0000 7f1c95735700 1 -- 172.21.15.43:0/95850533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c90159fd0 con 0x7f1c900b20e0 2023-06-28T20:07:34.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.260+0000 7f1c777fe700 1 -- 172.21.15.43:0/95850533 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c84009540 con 0x7f1c900b20e0 2023-06-28T20:07:34.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.260+0000 7f1c95735700 1 -- 172.21.15.43:0/95850533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c9015a360 con 0x7f1c900b20e0 2023-06-28T20:07:34.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.261+0000 7f1c95735700 1 -- 172.21.15.43:0/95850533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c900cc3e0 con 0x7f1c900b20e0 2023-06-28T20:07:34.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.261+0000 7f1c777fe700 1 -- 172.21.15.43:0/95850533 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1c840096a0 con 0x7f1c900b20e0 2023-06-28T20:07:34.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.262+0000 7f1c777fe700 1 --2- 172.21.15.43:0/95850533 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1c7c06ace0 0x7f1c7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.262+0000 7f1c777fe700 1 -- 172.21.15.43:0/95850533 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1c8408a9d0 con 0x7f1c900b20e0 2023-06-28T20:07:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.262+0000 7f1c8e7fc700 1 --2- 172.21.15.43:0/95850533 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1c7c06ace0 0x7f1c7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.262+0000 7f1c8e7fc700 1 --2- 172.21.15.43:0/95850533 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1c7c06ace0 0x7f1c7c06d190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1c900c5c50 tx=0x7f1c80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:34.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.266+0000 7f1c777fe700 1 -- 172.21.15.43:0/95850533 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1c84055a90 con 0x7f1c900b20e0 2023-06-28T20:07:34.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.465+0000 7f1c95735700 1 -- 172.21.15.43:0/95850533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c900728b0 con 0x7f1c900b20e0 2023-06-28T20:07:34.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.465+0000 7f1c777fe700 1 -- 172.21.15.43:0/95850533 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1c84019020 con 0x7f1c900b20e0 2023-06-28T20:07:34.469 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-06-28T20:07:34.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.467+0000 7f1c95735700 1 -- 172.21.15.43:0/95850533 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1c7c06ace0 msgr2=0x7f1c7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:34.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.467+0000 7f1c95735700 1 --2- 172.21.15.43:0/95850533 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1c7c06ace0 0x7f1c7c06d190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1c900c5c50 tx=0x7f1c80008040 comp rx=0 tx=0).stop 2023-06-28T20:07:34.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.467+0000 7f1c95735700 1 -- 172.21.15.43:0/95850533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c900b20e0 msgr2=0x7f1c90152c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:34.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.467+0000 7f1c95735700 1 --2- 172.21.15.43:0/95850533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c900b20e0 0x7f1c90152c00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400e730 tx=0x7f1c8400eaf0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.468+0000 7f1c95735700 1 -- 172.21.15.43:0/95850533 shutdown_connections 2023-06-28T20:07:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.468+0000 7f1c95735700 1 --2- 172.21.15.43:0/95850533 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1c7c06ace0 0x7f1c7c06d190 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.468+0000 7f1c95735700 1 --2- 172.21.15.43:0/95850533 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1c901596a0 0x7f1c9015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.468+0000 7f1c95735700 1 --2- 172.21.15.43:0/95850533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c900b20e0 0x7f1c90152c00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.468+0000 7f1c95735700 1 --2- 172.21.15.43:0/95850533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c901592d0 0x7f1c900b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.468+0000 7f1c95735700 1 -- 172.21.15.43:0/95850533 >> 172.21.15.43:0/95850533 conn(0x7f1c9006b000 msgr2=0x7f1c9006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.468+0000 7f1c95735700 1 -- 172.21.15.43:0/95850533 shutdown_connections 2023-06-28T20:07:34.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.468+0000 7f1c95735700 1 -- 172.21.15.43:0/95850533 wait complete. 2023-06-28T20:07:34.483 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-28T20:07:34.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.607+0000 7f472791c700 1 Processor -- start 2023-06-28T20:07:34.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.607+0000 7f472791c700 1 -- start start 2023-06-28T20:07:34.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.608+0000 7f472791c700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f47200aec90 0x7f4720160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:34.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.608+0000 7f472791c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4720160250 con 0x7f47200af060 2023-06-28T20:07:34.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.608+0000 7f472791c700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4720071410 con 0x7f472006ccc0 2023-06-28T20:07:34.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.608+0000 7f472791c700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f472015cb30 con 0x7f47200aec90 2023-06-28T20:07:34.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.608+0000 7f4724eb7700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f47200af060 0x7f4720070bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42736/0 (socket says 172.21.15.43:42736) 2023-06-28T20:07:34.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.608+0000 7f4724eb7700 1 -- 172.21.15.43:0/3561763846 learned_addr learned my addr 172.21.15.43:0/3561763846 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:34.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.608+0000 7f47256b8700 1 --2- 172.21.15.43:0/3561763846 >> v2:172.21.15.178:3300/0 conn(0x7f47200aec90 0x7f4720160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.608+0000 7f4717fff700 1 -- 172.21.15.43:0/3561763846 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1293175460 0 0) 0x7f4720160250 con 0x7f47200af060 2023-06-28T20:07:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.608+0000 7f4717fff700 1 -- 172.21.15.43:0/3561763846 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f470c0035b0 con 0x7f47200af060 2023-06-28T20:07:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.608+0000 7f4717fff700 1 -- 172.21.15.43:0/3561763846 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1237480915 0 0) 0x7f4720071410 con 0x7f472006ccc0 2023-06-28T20:07:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.608+0000 7f4717fff700 1 -- 172.21.15.43:0/3561763846 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4720160250 con 0x7f472006ccc0 2023-06-28T20:07:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.609+0000 7f47256b8700 1 -- 172.21.15.43:0/3561763846 >> v1:172.21.15.178:6789/0 conn(0x7f472006ccc0 legacy=0x7f47200b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.609+0000 7f4717fff700 1 -- 172.21.15.43:0/3561763846 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4247993205 0 0) 0x7f470c0035b0 con 0x7f47200af060 2023-06-28T20:07:34.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.609+0000 7f47256b8700 1 -- 172.21.15.43:0/3561763846 >> v1:172.21.15.43:6789/0 conn(0x7f47200af060 legacy=0x7f4720070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:34.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.609+0000 7f47256b8700 1 -- 172.21.15.43:0/3561763846 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47201604f0 con 0x7f47200aec90 2023-06-28T20:07:34.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.609+0000 7f47256b8700 1 --2- 172.21.15.43:0/3561763846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47200aec90 0x7f4720160cf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4710009840 tx=0x7f4710009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffde9288db564f32 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:34.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.609+0000 7f4717fff700 1 -- 172.21.15.43:0/3561763846 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c001670 con 0x7f47200aec90 2023-06-28T20:07:34.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.609+0000 7f4717fff700 1 -- 172.21.15.43:0/3561763846 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4710004030 con 0x7f47200aec90 2023-06-28T20:07:34.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.609+0000 7f4717fff700 1 -- 172.21.15.43:0/3561763846 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47100043c0 con 0x7f47200aec90 2023-06-28T20:07:34.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.609+0000 7f472791c700 1 -- 172.21.15.43:0/3561763846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47200aec90 msgr2=0x7f4720160cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:34.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.609+0000 7f472791c700 1 --2- 172.21.15.43:0/3561763846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47200aec90 0x7f4720160cf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4710009840 tx=0x7f4710009b50 comp rx=0 tx=0).stop 2023-06-28T20:07:34.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.610+0000 7f472791c700 1 -- 172.21.15.43:0/3561763846 shutdown_connections 2023-06-28T20:07:34.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.610+0000 7f472791c700 1 --2- 172.21.15.43:0/3561763846 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47200aec90 0x7f4720160cf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.610+0000 7f472791c700 1 -- 172.21.15.43:0/3561763846 >> 172.21.15.43:0/3561763846 conn(0x7f472006b000 msgr2=0x7f472006b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:34.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.610+0000 7f472791c700 1 -- 172.21.15.43:0/3561763846 shutdown_connections 2023-06-28T20:07:34.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.610+0000 7f472791c700 1 -- 172.21.15.43:0/3561763846 wait complete. 2023-06-28T20:07:34.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.611+0000 7f472791c700 1 Processor -- start 2023-06-28T20:07:34.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.611+0000 7f472791c700 1 -- start start 2023-06-28T20:07:34.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.612+0000 7f472791c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f472006ccc0 0x7f47200b5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:34.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.612+0000 7f472791c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47200aec90 0x7f47200ce840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:34.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.612+0000 7f472791c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47200af060 0x7f47200c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:34.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.612+0000 7f472791c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47200b4fe0 con 0x7f472006ccc0 2023-06-28T20:07:34.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.612+0000 7f472791c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47200b43d0 con 0x7f47200af060 2023-06-28T20:07:34.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.612+0000 7f472791c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47200b46b0 con 0x7f47200aec90 2023-06-28T20:07:34.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.612+0000 7f47256b8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f472006ccc0 0x7f47200b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:34.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.612+0000 7f47256b8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f472006ccc0 0x7f47200b5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53554/0 (socket says 172.21.15.43:53554) 2023-06-28T20:07:34.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.612+0000 7f47256b8700 1 -- 172.21.15.43:0/751115817 learned_addr learned my addr 172.21.15.43:0/751115817 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:34.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.612+0000 7f4724eb7700 1 --2- 172.21.15.43:0/751115817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47200aec90 0x7f47200ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:34.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.612+0000 7f4725eb9700 1 --2- 172.21.15.43:0/751115817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47200af060 0x7f47200c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:34.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.612+0000 7f47256b8700 1 -- 172.21.15.43:0/751115817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47200aec90 msgr2=0x7f47200ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:34.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.612+0000 7f47256b8700 1 --2- 172.21.15.43:0/751115817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47200aec90 0x7f47200ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.612+0000 7f47256b8700 1 -- 172.21.15.43:0/751115817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47200af060 msgr2=0x7f47200c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:34.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.612+0000 7f47256b8700 1 --2- 172.21.15.43:0/751115817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47200af060 0x7f47200c5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.612+0000 7f47256b8700 1 -- 172.21.15.43:0/751115817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47100094a0 con 0x7f472006ccc0 2023-06-28T20:07:34.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.612+0000 7f4724eb7700 1 --2- 172.21.15.43:0/751115817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47200aec90 0x7f47200ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:34.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.612+0000 7f4725eb9700 1 --2- 172.21.15.43:0/751115817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47200af060 0x7f47200c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:34.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.612+0000 7f47256b8700 1 --2- 172.21.15.43:0/751115817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f472006ccc0 0x7f47200b5470 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4710000c00 tx=0x7f4710004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:34.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.612+0000 7f4715ffb700 1 -- 172.21.15.43:0/751115817 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710004590 con 0x7f472006ccc0 2023-06-28T20:07:34.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.612+0000 7f4715ffb700 1 -- 172.21.15.43:0/751115817 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471001f070 con 0x7f472006ccc0 2023-06-28T20:07:34.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.612+0000 7f472791c700 1 -- 172.21.15.43:0/751115817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47200b4990 con 0x7f472006ccc0 2023-06-28T20:07:34.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.612+0000 7f4715ffb700 1 -- 172.21.15.43:0/751115817 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710011870 con 0x7f472006ccc0 2023-06-28T20:07:34.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.613+0000 7f472791c700 1 -- 172.21.15.43:0/751115817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47201517c0 con 0x7f472006ccc0 2023-06-28T20:07:34.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.614+0000 7f472791c700 1 -- 172.21.15.43:0/751115817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47200b52c0 con 0x7f472006ccc0 2023-06-28T20:07:34.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.614+0000 7f4715ffb700 1 -- 172.21.15.43:0/751115817 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f47100119d0 con 0x7f472006ccc0 2023-06-28T20:07:34.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.614+0000 7f4715ffb700 1 --2- 172.21.15.43:0/751115817 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f470406ace0 0x7f470406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:34.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.615+0000 7f4724eb7700 1 --2- 172.21.15.43:0/751115817 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f470406ace0 0x7f470406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:34.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.615+0000 7f4715ffb700 1 -- 172.21.15.43:0/751115817 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4710021070 con 0x7f472006ccc0 2023-06-28T20:07:34.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.615+0000 7f4724eb7700 1 --2- 172.21.15.43:0/751115817 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f470406ace0 0x7f470406d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f471c007fd0 tx=0x7f471c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:34.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.619+0000 7f4715ffb700 1 -- 172.21.15.43:0/751115817 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4710056140 con 0x7f472006ccc0 2023-06-28T20:07:34.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.817+0000 7f472791c700 1 -- 172.21.15.43:0/751115817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f47200728b0 con 0x7f472006ccc0 2023-06-28T20:07:34.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.818+0000 7f4715ffb700 1 -- 172.21.15.43:0/751115817 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4710059760 con 0x7f472006ccc0 2023-06-28T20:07:34.821 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-06-28T20:07:34.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.820+0000 7f472791c700 1 -- 172.21.15.43:0/751115817 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f470406ace0 msgr2=0x7f470406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:34.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.820+0000 7f472791c700 1 --2- 172.21.15.43:0/751115817 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f470406ace0 0x7f470406d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f471c007fd0 tx=0x7f471c009040 comp rx=0 tx=0).stop 2023-06-28T20:07:34.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.820+0000 7f472791c700 1 -- 172.21.15.43:0/751115817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f472006ccc0 msgr2=0x7f47200b5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:34.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.820+0000 7f472791c700 1 --2- 172.21.15.43:0/751115817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f472006ccc0 0x7f47200b5470 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4710000c00 tx=0x7f4710004750 comp rx=0 tx=0).stop 2023-06-28T20:07:34.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.820+0000 7f472791c700 1 -- 172.21.15.43:0/751115817 shutdown_connections 2023-06-28T20:07:34.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.820+0000 7f472791c700 1 --2- 172.21.15.43:0/751115817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47200af060 0x7f47200c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.820+0000 7f472791c700 1 --2- 172.21.15.43:0/751115817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f472006ccc0 0x7f47200b5470 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.820+0000 7f472791c700 1 --2- 172.21.15.43:0/751115817 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f470406ace0 0x7f470406d190 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.820+0000 7f472791c700 1 --2- 172.21.15.43:0/751115817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f47200aec90 0x7f47200ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.820+0000 7f472791c700 1 -- 172.21.15.43:0/751115817 >> 172.21.15.43:0/751115817 conn(0x7f472006b000 msgr2=0x7f472006d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:34.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.820+0000 7f472791c700 1 -- 172.21.15.43:0/751115817 shutdown_connections 2023-06-28T20:07:34.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.820+0000 7f472791c700 1 -- 172.21.15.43:0/751115817 wait complete. 2023-06-28T20:07:34.835 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-28T20:07:34.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.960+0000 7fbe51086700 1 Processor -- start 2023-06-28T20:07:34.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.960+0000 7fbe51086700 1 -- start start 2023-06-28T20:07:34.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.961+0000 7fbe51086700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbe4c1592d0 0x7fbe4c0b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:34.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.961+0000 7fbe51086700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbe4c1596a0 0x7fbe4c0b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:34.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.961+0000 7fbe51086700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe4c070b40 con 0x7fbe4c071900 2023-06-28T20:07:34.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.961+0000 7fbe51086700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe4c070e20 con 0x7fbe4c1596a0 2023-06-28T20:07:34.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.961+0000 7fbe51086700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe4c071100 con 0x7fbe4c1592d0 2023-06-28T20:07:34.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.961+0000 7fbe4a59c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbe4c1596a0 0x7fbe4c0b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:34.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.961+0000 7fbe4a59c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbe4c1596a0 0x7fbe4c0b1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53562/0 (socket says 172.21.15.43:53562) 2023-06-28T20:07:34.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.961+0000 7fbe4a59c700 1 -- 172.21.15.43:0/341531204 learned_addr learned my addr 172.21.15.43:0/341531204 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:34.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.961+0000 7fbe4ad9d700 1 --2- 172.21.15.43:0/341531204 >> v2:172.21.15.178:3300/0 conn(0x7fbe4c1592d0 0x7fbe4c0b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:34.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.961+0000 7fbe4a59c700 1 -- 172.21.15.43:0/341531204 >> v2:172.21.15.178:3300/0 conn(0x7fbe4c1592d0 msgr2=0x7fbe4c0b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:34.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.961+0000 7fbe4a59c700 1 --2- 172.21.15.43:0/341531204 >> v2:172.21.15.178:3300/0 conn(0x7fbe4c1592d0 0x7fbe4c0b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.961+0000 7fbe4a59c700 1 -- 172.21.15.43:0/341531204 >> v1:172.21.15.178:6789/0 conn(0x7fbe4c071900 legacy=0x7fbe4c15d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:34.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.961+0000 7fbe4a59c700 1 -- 172.21.15.43:0/341531204 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4c0713e0 con 0x7fbe4c1596a0 2023-06-28T20:07:34.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.961+0000 7fbe4a59c700 1 --2- 172.21.15.43:0/341531204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe4c1596a0 0x7fbe4c0b1f10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c0d0bf0 tx=0x7fbe3c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b0de668ef17861 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:34.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.962+0000 7fbe49d9b700 1 -- 172.21.15.43:0/341531204 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c004230 con 0x7fbe4c1596a0 2023-06-28T20:07:34.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.962+0000 7fbe49d9b700 1 -- 172.21.15.43:0/341531204 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3c004390 con 0x7fbe4c1596a0 2023-06-28T20:07:34.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.962+0000 7fbe49d9b700 1 -- 172.21.15.43:0/341531204 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c0036a0 con 0x7fbe4c1596a0 2023-06-28T20:07:34.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.962+0000 7fbe51086700 1 -- 172.21.15.43:0/341531204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe4c1596a0 msgr2=0x7fbe4c0b1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:34.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.962+0000 7fbe51086700 1 --2- 172.21.15.43:0/341531204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe4c1596a0 0x7fbe4c0b1f10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c0d0bf0 tx=0x7fbe3c009bd0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.962+0000 7fbe51086700 1 -- 172.21.15.43:0/341531204 shutdown_connections 2023-06-28T20:07:34.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.962+0000 7fbe51086700 1 --2- 172.21.15.43:0/341531204 >> v2:172.21.15.178:3300/0 conn(0x7fbe4c1592d0 0x7fbe4c0b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.962+0000 7fbe51086700 1 --2- 172.21.15.43:0/341531204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe4c1596a0 0x7fbe4c0b1f10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.962+0000 7fbe51086700 1 -- 172.21.15.43:0/341531204 >> 172.21.15.43:0/341531204 conn(0x7fbe4c06b000 msgr2=0x7fbe4c06b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.962+0000 7fbe51086700 1 -- 172.21.15.43:0/341531204 shutdown_connections 2023-06-28T20:07:34.978 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.962+0000 7fbe51086700 1 -- 172.21.15.43:0/341531204 wait complete. 2023-06-28T20:07:34.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.964+0000 7fbe51086700 1 Processor -- start 2023-06-28T20:07:34.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.964+0000 7fbe51086700 1 -- start start 2023-06-28T20:07:34.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.964+0000 7fbe51086700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe4c071900 0x7fbe4c0c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:34.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.964+0000 7fbe51086700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe4c1592d0 0x7fbe4c159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:34.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.964+0000 7fbe51086700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe4c1596a0 0x7fbe4c14d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:34.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.964+0000 7fbe51086700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c151e40 con 0x7fbe4c1592d0 2023-06-28T20:07:34.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.964+0000 7fbe51086700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c0c65d0 con 0x7fbe4c1596a0 2023-06-28T20:07:34.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.964+0000 7fbe51086700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4c15c340 con 0x7fbe4c071900 2023-06-28T20:07:34.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.965+0000 7fbe4a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe4c1592d0 0x7fbe4c159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:34.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.965+0000 7fbe4a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe4c1592d0 0x7fbe4c159be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53568/0 (socket says 172.21.15.43:53568) 2023-06-28T20:07:34.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.965+0000 7fbe4a59c700 1 -- 172.21.15.43:0/1990431686 learned_addr learned my addr 172.21.15.43:0/1990431686 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:34.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.965+0000 7fbe4ad9d700 1 --2- 172.21.15.43:0/1990431686 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe4c071900 0x7fbe4c0c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:34.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.965+0000 7fbe4b59e700 1 --2- 172.21.15.43:0/1990431686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe4c1596a0 0x7fbe4c14d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:34.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.965+0000 7fbe4a59c700 1 -- 172.21.15.43:0/1990431686 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe4c071900 msgr2=0x7fbe4c0c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:34.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.965+0000 7fbe4a59c700 1 --2- 172.21.15.43:0/1990431686 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe4c071900 0x7fbe4c0c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.965+0000 7fbe4a59c700 1 -- 172.21.15.43:0/1990431686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe4c1596a0 msgr2=0x7fbe4c14d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:34.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.965+0000 7fbe4a59c700 1 --2- 172.21.15.43:0/1990431686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe4c1596a0 0x7fbe4c14d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:34.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.965+0000 7fbe4a59c700 1 -- 172.21.15.43:0/1990431686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c009550 con 0x7fbe4c1592d0 2023-06-28T20:07:34.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.965+0000 7fbe4ad9d700 1 --2- 172.21.15.43:0/1990431686 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe4c071900 0x7fbe4c0c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.965+0000 7fbe4b59e700 1 --2- 172.21.15.43:0/1990431686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe4c1596a0 0x7fbe4c14d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.965+0000 7fbe4a59c700 1 --2- 172.21.15.43:0/1990431686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe4c1592d0 0x7fbe4c159be0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c009c10 tx=0x7fbe3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.965+0000 7fbe337fe700 1 -- 172.21.15.43:0/1990431686 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c011630 con 0x7fbe4c1592d0 2023-06-28T20:07:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.965+0000 7fbe337fe700 1 -- 172.21.15.43:0/1990431686 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3c011790 con 0x7fbe4c1592d0 2023-06-28T20:07:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.965+0000 7fbe51086700 1 -- 172.21.15.43:0/1990431686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe4c15c5e0 con 0x7fbe4c1592d0 2023-06-28T20:07:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.965+0000 7fbe337fe700 1 -- 172.21.15.43:0/1990431686 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c011b40 con 0x7fbe4c1592d0 2023-06-28T20:07:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.965+0000 7fbe51086700 1 -- 172.21.15.43:0/1990431686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe4c15c8c0 con 0x7fbe4c1592d0 2023-06-28T20:07:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.966+0000 7fbe51086700 1 -- 172.21.15.43:0/1990431686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe4c0cc3e0 con 0x7fbe4c1592d0 2023-06-28T20:07:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.967+0000 7fbe337fe700 1 -- 172.21.15.43:0/1990431686 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbe3c011ca0 con 0x7fbe4c1592d0 2023-06-28T20:07:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.967+0000 7fbe337fe700 1 --2- 172.21.15.43:0/1990431686 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbe3806ace0 0x7fbe3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:34.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.967+0000 7fbe337fe700 1 -- 172.21.15.43:0/1990431686 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbe3c08aa20 con 0x7fbe4c1592d0 2023-06-28T20:07:34.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.967+0000 7fbe4ad9d700 1 --2- 172.21.15.43:0/1990431686 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbe3806ace0 0x7fbe3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:34.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.967+0000 7fbe4ad9d700 1 --2- 172.21.15.43:0/1990431686 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbe3806ace0 0x7fbe3806d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbe400099f0 tx=0x7fbe40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:34.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:34.971+0000 7fbe337fe700 1 -- 172.21.15.43:0/1990431686 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbe3c055b60 con 0x7fbe4c1592d0 2023-06-28T20:07:35.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.168+0000 7fbe51086700 1 -- 172.21.15.43:0/1990431686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbe4c0728b0 con 0x7fbe4c1592d0 2023-06-28T20:07:35.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.168+0000 7fbe337fe700 1 -- 172.21.15.43:0/1990431686 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbe3c01f020 con 0x7fbe4c1592d0 2023-06-28T20:07:35.172 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-06-28T20:07:35.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.170+0000 7fbe51086700 1 -- 172.21.15.43:0/1990431686 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbe3806ace0 msgr2=0x7fbe3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:35.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.170+0000 7fbe51086700 1 --2- 172.21.15.43:0/1990431686 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbe3806ace0 0x7fbe3806d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbe400099f0 tx=0x7fbe40008040 comp rx=0 tx=0).stop 2023-06-28T20:07:35.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.170+0000 7fbe51086700 1 -- 172.21.15.43:0/1990431686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe4c1592d0 msgr2=0x7fbe4c159be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:35.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.170+0000 7fbe51086700 1 --2- 172.21.15.43:0/1990431686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe4c1592d0 0x7fbe4c159be0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c009c10 tx=0x7fbe3c004750 comp rx=0 tx=0).stop 2023-06-28T20:07:35.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.170+0000 7fbe51086700 1 -- 172.21.15.43:0/1990431686 shutdown_connections 2023-06-28T20:07:35.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.170+0000 7fbe51086700 1 --2- 172.21.15.43:0/1990431686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe4c1596a0 0x7fbe4c14d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:35.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.170+0000 7fbe51086700 1 --2- 172.21.15.43:0/1990431686 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbe3806ace0 0x7fbe3806d190 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:35.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.170+0000 7fbe51086700 1 --2- 172.21.15.43:0/1990431686 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe4c071900 0x7fbe4c0c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:35.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.170+0000 7fbe51086700 1 --2- 172.21.15.43:0/1990431686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe4c1592d0 0x7fbe4c159be0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:35.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.170+0000 7fbe51086700 1 -- 172.21.15.43:0/1990431686 >> 172.21.15.43:0/1990431686 conn(0x7fbe4c06b000 msgr2=0x7fbe4c069ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:35.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.170+0000 7fbe51086700 1 -- 172.21.15.43:0/1990431686 shutdown_connections 2023-06-28T20:07:35.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.170+0000 7fbe51086700 1 -- 172.21.15.43:0/1990431686 wait complete. 2023-06-28T20:07:35.186 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-28T20:07:35.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.308+0000 7fc6f5e3c700 1 Processor -- start 2023-06-28T20:07:35.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.308+0000 7fc6f5e3c700 1 -- start start 2023-06-28T20:07:35.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.309+0000 7fc6f5e3c700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc6f00d10f0 0x7fc6f00ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:35.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.309+0000 7fc6f5e3c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f015ab50 con 0x7fc6f00cbab0 2023-06-28T20:07:35.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.309+0000 7fc6f5e3c700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f00cd1b0 con 0x7fc6f00cedf0 2023-06-28T20:07:35.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.309+0000 7fc6f5e3c700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f00cbf30 con 0x7fc6f00d10f0 2023-06-28T20:07:35.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.309+0000 7fc6effff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc6f00cbab0 0x7fc6f00b0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42764/0 (socket says 172.21.15.43:42764) 2023-06-28T20:07:35.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.309+0000 7fc6effff700 1 -- 172.21.15.43:0/3680871848 learned_addr learned my addr 172.21.15.43:0/3680871848 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:35.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.309+0000 7fc6ef7fe700 1 --2- 172.21.15.43:0/3680871848 >> v2:172.21.15.178:3300/0 conn(0x7fc6f00d10f0 0x7fc6f00ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:35.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.309+0000 7fc6ee7fc700 1 -- 172.21.15.43:0/3680871848 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4259678533 0 0) 0x7fc6f015ab50 con 0x7fc6f00cbab0 2023-06-28T20:07:35.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.309+0000 7fc6ee7fc700 1 -- 172.21.15.43:0/3680871848 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6dc0035b0 con 0x7fc6f00cbab0 2023-06-28T20:07:35.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.310+0000 7fc6ef7fe700 1 -- 172.21.15.43:0/3680871848 >> v1:172.21.15.178:6789/0 conn(0x7fc6f00cedf0 legacy=0x7fc6f00cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:35.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.310+0000 7fc6ef7fe700 1 -- 172.21.15.43:0/3680871848 >> v1:172.21.15.43:6789/0 conn(0x7fc6f00cbab0 legacy=0x7fc6f00b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:35.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.310+0000 7fc6ef7fe700 1 -- 172.21.15.43:0/3680871848 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f00cc110 con 0x7fc6f00d10f0 2023-06-28T20:07:35.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.310+0000 7fc6ef7fe700 1 --2- 172.21.15.43:0/3680871848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6f00d10f0 0x7fc6f00ce8b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009820 tx=0x7fc6e0009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a87698ab6276284 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:35.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.310+0000 7fc6ee7fc700 1 -- 172.21.15.43:0/3680871848 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0005e30 con 0x7fc6f00d10f0 2023-06-28T20:07:35.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.310+0000 7fc6ee7fc700 1 -- 172.21.15.43:0/3680871848 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e0006030 con 0x7fc6f00d10f0 2023-06-28T20:07:35.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.310+0000 7fc6ee7fc700 1 -- 172.21.15.43:0/3680871848 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e00063c0 con 0x7fc6f00d10f0 2023-06-28T20:07:35.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.310+0000 7fc6f5e3c700 1 -- 172.21.15.43:0/3680871848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6f00d10f0 msgr2=0x7fc6f00ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:35.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.310+0000 7fc6f5e3c700 1 --2- 172.21.15.43:0/3680871848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6f00d10f0 0x7fc6f00ce8b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009820 tx=0x7fc6e0009b30 comp rx=0 tx=0).stop 2023-06-28T20:07:35.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.310+0000 7fc6f5e3c700 1 -- 172.21.15.43:0/3680871848 shutdown_connections 2023-06-28T20:07:35.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.310+0000 7fc6f5e3c700 1 --2- 172.21.15.43:0/3680871848 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6f00d10f0 0x7fc6f00ce8b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:35.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.310+0000 7fc6f5e3c700 1 -- 172.21.15.43:0/3680871848 >> 172.21.15.43:0/3680871848 conn(0x7fc6f00664b0 msgr2=0x7fc6f009a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:35.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.311+0000 7fc6f5e3c700 1 -- 172.21.15.43:0/3680871848 shutdown_connections 2023-06-28T20:07:35.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.311+0000 7fc6f5e3c700 1 -- 172.21.15.43:0/3680871848 wait complete. 2023-06-28T20:07:35.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.312+0000 7fc6f5e3c700 1 Processor -- start 2023-06-28T20:07:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.312+0000 7fc6f5e3c700 1 -- start start 2023-06-28T20:07:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.313+0000 7fc6f5e3c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6f00cbab0 0x7fc6f0159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.313+0000 7fc6f5e3c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f00cedf0 0x7fc6f00b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:35.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.313+0000 7fc6f5e3c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6f00d10f0 0x7fc6f009cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.313+0000 7fc6f5e3c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f009e660 con 0x7fc6f00cedf0 2023-06-28T20:07:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.313+0000 7fc6f5e3c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f009dd00 con 0x7fc6f00d10f0 2023-06-28T20:07:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.313+0000 7fc6f5e3c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f009c1b0 con 0x7fc6f00cbab0 2023-06-28T20:07:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.313+0000 7fc6eeffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f00cedf0 0x7fc6f00b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.313+0000 7fc6eeffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f00cedf0 0x7fc6f00b76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53582/0 (socket says 172.21.15.43:53582) 2023-06-28T20:07:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.313+0000 7fc6eeffd700 1 -- 172.21.15.43:0/1383507928 learned_addr learned my addr 172.21.15.43:0/1383507928 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:35.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.313+0000 7fc6effff700 1 --2- 172.21.15.43:0/1383507928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6f00d10f0 0x7fc6f009cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:35.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.313+0000 7fc6ef7fe700 1 --2- 172.21.15.43:0/1383507928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6f00cbab0 0x7fc6f0159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:35.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.313+0000 7fc6eeffd700 1 -- 172.21.15.43:0/1383507928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6f00cbab0 msgr2=0x7fc6f0159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:35.323 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.313+0000 7fc6eeffd700 1 --2- 172.21.15.43:0/1383507928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6f00cbab0 0x7fc6f0159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:35.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.313+0000 7fc6eeffd700 1 -- 172.21.15.43:0/1383507928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6f00d10f0 msgr2=0x7fc6f009cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:35.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.313+0000 7fc6eeffd700 1 --2- 172.21.15.43:0/1383507928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6f00d10f0 0x7fc6f009cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:35.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.313+0000 7fc6eeffd700 1 -- 172.21.15.43:0/1383507928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e0009430 con 0x7fc6f00cedf0 2023-06-28T20:07:35.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.313+0000 7fc6effff700 1 --2- 172.21.15.43:0/1383507928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6f00d10f0 0x7fc6f009cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:35.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.313+0000 7fc6eeffd700 1 --2- 172.21.15.43:0/1383507928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f00cedf0 0x7fc6f00b76c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800c510 tx=0x7fc6d800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:35.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.313+0000 7fc6e7fff700 1 -- 172.21.15.43:0/1383507928 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d80052d0 con 0x7fc6f00cedf0 2023-06-28T20:07:35.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.313+0000 7fc6e7fff700 1 -- 172.21.15.43:0/1383507928 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d8005450 con 0x7fc6f00cedf0 2023-06-28T20:07:35.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.313+0000 7fc6f5e3c700 1 -- 172.21.15.43:0/1383507928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f00c7ee0 con 0x7fc6f00cedf0 2023-06-28T20:07:35.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.313+0000 7fc6e7fff700 1 -- 172.21.15.43:0/1383507928 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d80038c0 con 0x7fc6f00cedf0 2023-06-28T20:07:35.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.313+0000 7fc6f5e3c700 1 -- 172.21.15.43:0/1383507928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f00c84a0 con 0x7fc6f00cedf0 2023-06-28T20:07:35.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.314+0000 7fc6f5e3c700 1 -- 172.21.15.43:0/1383507928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f00cfd40 con 0x7fc6f00cedf0 2023-06-28T20:07:35.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.315+0000 7fc6e7fff700 1 -- 172.21.15.43:0/1383507928 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc6d8003a20 con 0x7fc6f00cedf0 2023-06-28T20:07:35.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.315+0000 7fc6e7fff700 1 --2- 172.21.15.43:0/1383507928 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc6d006ace0 0x7fc6d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:35.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.315+0000 7fc6e7fff700 1 -- 172.21.15.43:0/1383507928 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc6d80888f0 con 0x7fc6f00cedf0 2023-06-28T20:07:35.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.315+0000 7fc6ef7fe700 1 --2- 172.21.15.43:0/1383507928 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc6d006ace0 0x7fc6d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:35.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.316+0000 7fc6ef7fe700 1 --2- 172.21.15.43:0/1383507928 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc6d006ace0 0x7fc6d006d190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009b70 tx=0x7fc6e0008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:35.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.320+0000 7fc6e7fff700 1 -- 172.21.15.43:0/1383507928 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc6d8053a30 con 0x7fc6f00cedf0 2023-06-28T20:07:35.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.561+0000 7fc6f5e3c700 1 -- 172.21.15.43:0/1383507928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6f00728b0 con 0x7fc6f00cedf0 2023-06-28T20:07:35.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.562+0000 7fc6e7fff700 1 -- 172.21.15.43:0/1383507928 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4665 (secure 0 0 0) 0x7fc6d8057050 con 0x7fc6f00cedf0 2023-06-28T20:07:35.565 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:35.565 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4033115951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4033115951},{"type":"v1","addr":"172.21.15.178:6835","nonce":4033115951}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/65569869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":65569869},{"type":"v1","addr":"172.21.15.43:6837","nonce":65569869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/1016213057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016213057},{"type":"v1","addr":"172.21.15.178:6834","nonce":1016213057}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-28T20:07:20.788417+0000","modified":"2023-06-28T20:07:24.388714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/3514544484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3514544484},{"type":"v1","addr":"172.21.15.43:6835","nonce":3514544484}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-28T20:07:35.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.564+0000 7fc6f5e3c700 1 -- 172.21.15.43:0/1383507928 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc6d006ace0 msgr2=0x7fc6d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:35.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.564+0000 7fc6f5e3c700 1 --2- 172.21.15.43:0/1383507928 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc6d006ace0 0x7fc6d006d190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009b70 tx=0x7fc6e0008070 comp rx=0 tx=0).stop 2023-06-28T20:07:35.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.564+0000 7fc6f5e3c700 1 -- 172.21.15.43:0/1383507928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f00cedf0 msgr2=0x7fc6f00b76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:35.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.564+0000 7fc6f5e3c700 1 --2- 172.21.15.43:0/1383507928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f00cedf0 0x7fc6f00b76c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800c510 tx=0x7fc6d800c8d0 comp rx=0 tx=0).stop 2023-06-28T20:07:35.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.564+0000 7fc6f5e3c700 1 -- 172.21.15.43:0/1383507928 shutdown_connections 2023-06-28T20:07:35.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.564+0000 7fc6f5e3c700 1 --2- 172.21.15.43:0/1383507928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6f00d10f0 0x7fc6f009cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:35.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.564+0000 7fc6f5e3c700 1 --2- 172.21.15.43:0/1383507928 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc6d006ace0 0x7fc6d006d190 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:35.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.564+0000 7fc6f5e3c700 1 --2- 172.21.15.43:0/1383507928 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc6f00cbab0 0x7fc6f0159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:35.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.564+0000 7fc6f5e3c700 1 --2- 172.21.15.43:0/1383507928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6f00cedf0 0x7fc6f00b76c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:35.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.564+0000 7fc6f5e3c700 1 -- 172.21.15.43:0/1383507928 >> 172.21.15.43:0/1383507928 conn(0x7fc6f00664b0 msgr2=0x7fc6f00afb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:35.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.564+0000 7fc6f5e3c700 1 -- 172.21.15.43:0/1383507928 shutdown_connections 2023-06-28T20:07:35.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.564+0000 7fc6f5e3c700 1 -- 172.21.15.43:0/1383507928 wait complete. 2023-06-28T20:07:35.569 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 7 2023-06-28T20:07:35.579 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:07:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.701+0000 7f52184c5700 1 Processor -- start 2023-06-28T20:07:35.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.701+0000 7f52184c5700 1 -- start start 2023-06-28T20:07:35.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.702+0000 7f52184c5700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5210159820 0x7f521015a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:35.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.702+0000 7f52184c5700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f521006ee20 0x7f52100cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:35.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.702+0000 7f52184c5700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52100ce0a0 con 0x7f52100cc600 2023-06-28T20:07:35.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.702+0000 7f52184c5700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52100ce380 con 0x7f521006ee20 2023-06-28T20:07:35.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.702+0000 7f52184c5700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5210159c90 con 0x7f5210159820 2023-06-28T20:07:35.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.702+0000 7f5216261700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f52100cc600 0x7f52100cc9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42774/0 (socket says 172.21.15.43:42774) 2023-06-28T20:07:35.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.702+0000 7f5216261700 1 -- 172.21.15.43:0/211268247 learned_addr learned my addr 172.21.15.43:0/211268247 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:35.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f5216a62700 1 --2- 172.21.15.43:0/211268247 >> v2:172.21.15.178:3300/0 conn(0x7f521006ee20 0x7f52100cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:35.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f5215a60700 1 --2- 172.21.15.43:0/211268247 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5210159820 0x7f521015a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:35.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f521525f700 1 -- 172.21.15.43:0/211268247 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 860701293 0 0) 0x7f52100ce0a0 con 0x7f52100cc600 2023-06-28T20:07:35.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f521525f700 1 -- 172.21.15.43:0/211268247 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51fc0035b0 con 0x7f52100cc600 2023-06-28T20:07:35.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f521525f700 1 -- 172.21.15.43:0/211268247 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1342519483 0 0) 0x7f51fc0035b0 con 0x7f52100cc600 2023-06-28T20:07:35.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f521525f700 1 -- 172.21.15.43:0/211268247 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5210159820 msgr2=0x7f521015a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:35.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f521525f700 1 --2- 172.21.15.43:0/211268247 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5210159820 0x7f521015a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:35.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f521525f700 1 -- 172.21.15.43:0/211268247 >> v2:172.21.15.178:3300/0 conn(0x7f521006ee20 msgr2=0x7f52100cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:35.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f521525f700 1 --2- 172.21.15.43:0/211268247 >> v2:172.21.15.178:3300/0 conn(0x7f521006ee20 0x7f52100cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:35.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f521525f700 1 -- 172.21.15.43:0/211268247 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5210159f70 con 0x7f52100cc600 2023-06-28T20:07:35.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f521525f700 1 -- 172.21.15.43:0/211268247 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f5200001670 con 0x7f52100cc600 2023-06-28T20:07:35.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f521525f700 1 -- 172.21.15.43:0/211268247 >> v1:172.21.15.43:6789/0 conn(0x7f52100cc600 legacy=0x7f52100cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:35.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f521525f700 1 --2- 172.21.15.43:0/211268247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51fc003a00 0x7f51fc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:35.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f521525f700 1 --2- 172.21.15.43:0/211268247 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5210159820 0x7f51fc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:35.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f521525f700 1 --2- 172.21.15.43:0/211268247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51fc004d90 0x7f51fc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:35.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f521525f700 1 -- 172.21.15.43:0/211268247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52100ce380 con 0x7f51fc003a00 2023-06-28T20:07:35.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f521525f700 1 -- 172.21.15.43:0/211268247 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5210159c90 con 0x7f51fc004d90 2023-06-28T20:07:35.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f521525f700 1 -- 172.21.15.43:0/211268247 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc00ba10 con 0x7f5210159820 2023-06-28T20:07:35.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f5216261700 1 --2- 172.21.15.43:0/211268247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51fc003a00 0x7f51fc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:35.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f5216a62700 1 --2- 172.21.15.43:0/211268247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51fc004d90 0x7f51fc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:35.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f5215a60700 1 --2- 172.21.15.43:0/211268247 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5210159820 0x7f51fc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:35.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f5216261700 1 -- 172.21.15.43:0/211268247 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5210159820 msgr2=0x7f51fc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:35.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f5216261700 1 --2- 172.21.15.43:0/211268247 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5210159820 0x7f51fc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:35.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f5216261700 1 -- 172.21.15.43:0/211268247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51fc004d90 msgr2=0x7f51fc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:35.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f5216261700 1 --2- 172.21.15.43:0/211268247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51fc004d90 0x7f51fc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:35.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.703+0000 7f5216261700 1 -- 172.21.15.43:0/211268247 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc0035b0 con 0x7f51fc003a00 2023-06-28T20:07:35.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.704+0000 7f5216261700 1 --2- 172.21.15.43:0/211268247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51fc003a00 0x7f51fc003e10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f520000c320 tx=0x7f51fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af8f42230e7aaa server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:35.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.704+0000 7f521525f700 1 -- 172.21.15.43:0/211268247 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520000ce60 con 0x7f51fc003a00 2023-06-28T20:07:35.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.704+0000 7f521525f700 1 -- 172.21.15.43:0/211268247 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5200005960 con 0x7f51fc003a00 2023-06-28T20:07:35.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.704+0000 7f521525f700 1 -- 172.21.15.43:0/211268247 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5200005cf0 con 0x7f51fc003a00 2023-06-28T20:07:35.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.704+0000 7f52184c5700 1 -- 172.21.15.43:0/211268247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51fc003a00 msgr2=0x7f51fc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:35.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.704+0000 7f52184c5700 1 --2- 172.21.15.43:0/211268247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51fc003a00 0x7f51fc003e10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f520000c320 tx=0x7f51fc005880 comp rx=0 tx=0).stop 2023-06-28T20:07:35.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.704+0000 7f5216a62700 1 -- 172.21.15.43:0/211268247 reap_dead start 2023-06-28T20:07:35.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.704+0000 7f52184c5700 1 -- 172.21.15.43:0/211268247 shutdown_connections 2023-06-28T20:07:35.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.704+0000 7f52184c5700 1 -- 172.21.15.43:0/211268247 >> 172.21.15.43:0/211268247 conn(0x7f52100b1720 msgr2=0x7f52100b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:35.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.704+0000 7f52184c5700 1 -- 172.21.15.43:0/211268247 shutdown_connections 2023-06-28T20:07:35.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.704+0000 7f52184c5700 1 -- 172.21.15.43:0/211268247 wait complete. 2023-06-28T20:07:35.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.706+0000 7f52184c5700 1 Processor -- start 2023-06-28T20:07:35.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.706+0000 7f52184c5700 1 -- start start 2023-06-28T20:07:35.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.706+0000 7f52184c5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51fc003a00 0x7f52100cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:35.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.706+0000 7f52184c5700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52101ee540 0x7f52101ee9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:35.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.706+0000 7f52184c5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52100c5370 0x7f52100c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:35.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.706+0000 7f52184c5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5210143220 con 0x7f52100c5370 2023-06-28T20:07:35.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.706+0000 7f52184c5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52100c60e0 con 0x7f51fc003a00 2023-06-28T20:07:35.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.706+0000 7f52184c5700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52100c63c0 con 0x7f52101ee540 2023-06-28T20:07:35.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.706+0000 7f5216a62700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52100c5370 0x7f52100c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:35.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.707+0000 7f5216a62700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52100c5370 0x7f52100c57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53604/0 (socket says 172.21.15.43:53604) 2023-06-28T20:07:35.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.707+0000 7f5216a62700 1 -- 172.21.15.43:0/4165683706 learned_addr learned my addr 172.21.15.43:0/4165683706 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:35.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.707+0000 7f5216261700 1 --2- 172.21.15.43:0/4165683706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51fc003a00 0x7f52100cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:35.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.707+0000 7f5215a60700 1 --2- 172.21.15.43:0/4165683706 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52101ee540 0x7f52101ee9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:35.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.707+0000 7f5216a62700 1 -- 172.21.15.43:0/4165683706 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52101ee540 msgr2=0x7f52101ee9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:35.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.707+0000 7f5216a62700 1 --2- 172.21.15.43:0/4165683706 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52101ee540 0x7f52101ee9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:35.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.707+0000 7f5216a62700 1 -- 172.21.15.43:0/4165683706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51fc003a00 msgr2=0x7f52100cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:35.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.707+0000 7f5216a62700 1 --2- 172.21.15.43:0/4165683706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51fc003a00 0x7f52100cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:35.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.707+0000 7f5216a62700 1 -- 172.21.15.43:0/4165683706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5210143470 con 0x7f52100c5370 2023-06-28T20:07:35.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.707+0000 7f5216a62700 1 --2- 172.21.15.43:0/4165683706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52100c5370 0x7f52100c57e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f52100cd480 tx=0x7f5208011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:35.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.707+0000 7f5206ffd700 1 -- 172.21.15.43:0/4165683706 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520800e040 con 0x7f52100c5370 2023-06-28T20:07:35.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.707+0000 7f5206ffd700 1 -- 172.21.15.43:0/4165683706 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f520800f040 con 0x7f52100c5370 2023-06-28T20:07:35.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.707+0000 7f52184c5700 1 -- 172.21.15.43:0/4165683706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5210143750 con 0x7f52100c5370 2023-06-28T20:07:35.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.707+0000 7f5206ffd700 1 -- 172.21.15.43:0/4165683706 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520800fdb0 con 0x7f52100c5370 2023-06-28T20:07:35.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.707+0000 7f52184c5700 1 -- 172.21.15.43:0/4165683706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52100c5d20 con 0x7f52100c5370 2023-06-28T20:07:35.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.708+0000 7f52184c5700 1 -- 172.21.15.43:0/4165683706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52100c6580 con 0x7f52100c5370 2023-06-28T20:07:35.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.709+0000 7f5206ffd700 1 -- 172.21.15.43:0/4165683706 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5208022460 con 0x7f52100c5370 2023-06-28T20:07:35.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.709+0000 7f5206ffd700 1 --2- 172.21.15.43:0/4165683706 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f51f406ad10 0x7f51f406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:35.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.710+0000 7f5206ffd700 1 -- 172.21.15.43:0/4165683706 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f520808a730 con 0x7f52100c5370 2023-06-28T20:07:35.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.710+0000 7f5216261700 1 --2- 172.21.15.43:0/4165683706 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f51f406ad10 0x7f51f406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:35.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.710+0000 7f5216261700 1 --2- 172.21.15.43:0/4165683706 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f51f406ad10 0x7f51f406d1c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5200008fd0 tx=0x7f520000a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:35.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.713+0000 7f5206ffd700 1 -- 172.21.15.43:0/4165683706 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5208055730 con 0x7f52100c5370 2023-06-28T20:07:35.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.910+0000 7f52184c5700 1 -- 172.21.15.43:0/4165683706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52100728b0 con 0x7f52100c5370 2023-06-28T20:07:35.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.911+0000 7f5206ffd700 1 -- 172.21.15.43:0/4165683706 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18221 (secure 0 0 0) 0x7f5208058d50 con 0x7f52100c5370 2023-06-28T20:07:35.915 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:35.915 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:07:22.377248+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:07:18.624759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-28T20:07:19.760017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:07:35.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.913+0000 7f52184c5700 1 -- 172.21.15.43:0/4165683706 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f51f406ad10 msgr2=0x7f51f406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:35.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.913+0000 7f52184c5700 1 --2- 172.21.15.43:0/4165683706 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f51f406ad10 0x7f51f406d1c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5200008fd0 tx=0x7f520000a000 comp rx=0 tx=0).stop 2023-06-28T20:07:35.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.913+0000 7f52184c5700 1 -- 172.21.15.43:0/4165683706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52100c5370 msgr2=0x7f52100c57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:35.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.913+0000 7f52184c5700 1 --2- 172.21.15.43:0/4165683706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52100c5370 0x7f52100c57e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f52100cd480 tx=0x7f5208011f80 comp rx=0 tx=0).stop 2023-06-28T20:07:35.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.914+0000 7f52184c5700 1 -- 172.21.15.43:0/4165683706 shutdown_connections 2023-06-28T20:07:35.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.914+0000 7f52184c5700 1 --2- 172.21.15.43:0/4165683706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52100c5370 0x7f52100c57e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:35.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.914+0000 7f52184c5700 1 --2- 172.21.15.43:0/4165683706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51fc003a00 0x7f52100cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:35.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.914+0000 7f52184c5700 1 --2- 172.21.15.43:0/4165683706 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f51f406ad10 0x7f51f406d1c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:35.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.914+0000 7f52184c5700 1 --2- 172.21.15.43:0/4165683706 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f52101ee540 0x7f52101ee9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:35.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.914+0000 7f52184c5700 1 -- 172.21.15.43:0/4165683706 >> 172.21.15.43:0/4165683706 conn(0x7f52100b1720 msgr2=0x7f521015b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:35.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.914+0000 7f52184c5700 1 -- 172.21.15.43:0/4165683706 shutdown_connections 2023-06-28T20:07:35.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:35.914+0000 7f52184c5700 1 -- 172.21.15.43:0/4165683706 wait complete. 2023-06-28T20:07:35.929 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-28T20:07:36.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.051+0000 7f8dbf357700 1 Processor -- start 2023-06-28T20:07:36.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.051+0000 7f8dbf357700 1 -- start start 2023-06-28T20:07:36.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.052+0000 7f8dbf357700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8db80cb1f0 0x7f8db815b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.052+0000 7f8dbf357700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8db80cb5c0 0x7f8db80cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.052+0000 7f8dbf357700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8db806b230 0x7f8db80cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.052+0000 7f8dbf357700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db80cc740 con 0x7f8db80cb5c0 2023-06-28T20:07:36.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.052+0000 7f8dbf357700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db815e040 con 0x7f8db80cb1f0 2023-06-28T20:07:36.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.052+0000 7f8dbf357700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db815f010 con 0x7f8db806b230 2023-06-28T20:07:36.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.052+0000 7f8dbc8f2700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8db80cb5c0 0x7f8db80cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.052+0000 7f8dbc8f2700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8db80cb5c0 0x7f8db80cbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53608/0 (socket says 172.21.15.43:53608) 2023-06-28T20:07:36.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.052+0000 7f8dbc8f2700 1 -- 172.21.15.43:0/2764545880 learned_addr learned my addr 172.21.15.43:0/2764545880 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:36.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.052+0000 7f8dbd0f3700 1 --2- 172.21.15.43:0/2764545880 >> v2:172.21.15.178:3300/0 conn(0x7f8db80cb1f0 0x7f8db815b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.052+0000 7f8dbd8f4700 1 --2- 172.21.15.43:0/2764545880 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8db806b230 0x7f8db80cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.053+0000 7f8dbc8f2700 1 -- 172.21.15.43:0/2764545880 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8db806b230 msgr2=0x7f8db80cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:36.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.053+0000 7f8dbc8f2700 1 --2- 172.21.15.43:0/2764545880 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8db806b230 0x7f8db80cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.053+0000 7f8dbc8f2700 1 -- 172.21.15.43:0/2764545880 >> v2:172.21.15.178:3300/0 conn(0x7f8db80cb1f0 msgr2=0x7f8db815b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:36.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.053+0000 7f8dbc8f2700 1 --2- 172.21.15.43:0/2764545880 >> v2:172.21.15.178:3300/0 conn(0x7f8db80cb1f0 0x7f8db815b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.053+0000 7f8dbc8f2700 1 -- 172.21.15.43:0/2764545880 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db815e250 con 0x7f8db80cb5c0 2023-06-28T20:07:36.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.053+0000 7f8dbc8f2700 1 --2- 172.21.15.43:0/2764545880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8db80cb5c0 0x7f8db80cbcc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8db80cce30 tx=0x7f8db4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e03b8b454c1091 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:36.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.053+0000 7f8daffff700 1 -- 172.21.15.43:0/2764545880 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db4011830 con 0x7f8db80cb5c0 2023-06-28T20:07:36.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.053+0000 7f8daffff700 1 -- 172.21.15.43:0/2764545880 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db4011990 con 0x7f8db80cb5c0 2023-06-28T20:07:36.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.053+0000 7f8daffff700 1 -- 172.21.15.43:0/2764545880 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db4011d20 con 0x7f8db80cb5c0 2023-06-28T20:07:36.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.053+0000 7f8dbf357700 1 -- 172.21.15.43:0/2764545880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8db80cb5c0 msgr2=0x7f8db80cbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:36.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.053+0000 7f8dbf357700 1 --2- 172.21.15.43:0/2764545880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8db80cb5c0 0x7f8db80cbcc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8db80cce30 tx=0x7f8db4009c00 comp rx=0 tx=0).stop 2023-06-28T20:07:36.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.054+0000 7f8dbf357700 1 -- 172.21.15.43:0/2764545880 shutdown_connections 2023-06-28T20:07:36.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.054+0000 7f8dbf357700 1 --2- 172.21.15.43:0/2764545880 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8db806b230 0x7f8db80cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.054+0000 7f8dbf357700 1 --2- 172.21.15.43:0/2764545880 >> v2:172.21.15.178:3300/0 conn(0x7f8db80cb1f0 0x7f8db815b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.054+0000 7f8dbf357700 1 --2- 172.21.15.43:0/2764545880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8db80cb5c0 0x7f8db80cbcc0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.054+0000 7f8dbf357700 1 -- 172.21.15.43:0/2764545880 >> 172.21.15.43:0/2764545880 conn(0x7f8db806c170 msgr2=0x7f8db806c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:36.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.054+0000 7f8dbf357700 1 -- 172.21.15.43:0/2764545880 shutdown_connections 2023-06-28T20:07:36.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.054+0000 7f8dbf357700 1 -- 172.21.15.43:0/2764545880 wait complete. 2023-06-28T20:07:36.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.055+0000 7f8dbf357700 1 Processor -- start 2023-06-28T20:07:36.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.055+0000 7f8dbf357700 1 -- start start 2023-06-28T20:07:36.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.056+0000 7f8dbf357700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8db806b230 0x7f8db80a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.056+0000 7f8dbf357700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8db80cb1f0 0x7f8db81f22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.056+0000 7f8dbf357700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8db80cb5c0 0x7f8db80a9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.056+0000 7f8dbf357700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db8159ee0 con 0x7f8db80cb5c0 2023-06-28T20:07:36.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.056+0000 7f8dbf357700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db80cd400 con 0x7f8db80cb1f0 2023-06-28T20:07:36.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.056+0000 7f8dbf357700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db80cd6e0 con 0x7f8db806b230 2023-06-28T20:07:36.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.056+0000 7f8dbd8f4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8db80cb5c0 0x7f8db80a9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.056+0000 7f8dbd8f4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8db80cb5c0 0x7f8db80a9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53616/0 (socket says 172.21.15.43:53616) 2023-06-28T20:07:36.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.056+0000 7f8dbd8f4700 1 -- 172.21.15.43:0/195666706 learned_addr learned my addr 172.21.15.43:0/195666706 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:36.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.056+0000 7f8dbc8f2700 1 --2- 172.21.15.43:0/195666706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8db80cb1f0 0x7f8db81f22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.056+0000 7f8dbd0f3700 1 --2- 172.21.15.43:0/195666706 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8db806b230 0x7f8db80a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.056+0000 7f8dbd8f4700 1 -- 172.21.15.43:0/195666706 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8db806b230 msgr2=0x7f8db80a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:36.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.056+0000 7f8dbd8f4700 1 --2- 172.21.15.43:0/195666706 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8db806b230 0x7f8db80a8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.056+0000 7f8dbd8f4700 1 -- 172.21.15.43:0/195666706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8db80cb1f0 msgr2=0x7f8db81f22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:36.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.056+0000 7f8dbd8f4700 1 --2- 172.21.15.43:0/195666706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8db80cb1f0 0x7f8db81f22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.056+0000 7f8dbd8f4700 1 -- 172.21.15.43:0/195666706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db4009550 con 0x7f8db80cb5c0 2023-06-28T20:07:36.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.056+0000 7f8dbc8f2700 1 --2- 172.21.15.43:0/195666706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8db80cb1f0 0x7f8db81f22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:36.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.056+0000 7f8dbd8f4700 1 --2- 172.21.15.43:0/195666706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8db80cb5c0 0x7f8db80a9060 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8db000eed0 tx=0x7f8db000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:36.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.056+0000 7f8dadffb700 1 -- 172.21.15.43:0/195666706 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db0010040 con 0x7f8db80cb5c0 2023-06-28T20:07:36.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.056+0000 7f8dadffb700 1 -- 172.21.15.43:0/195666706 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db000bd30 con 0x7f8db80cb5c0 2023-06-28T20:07:36.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.056+0000 7f8dbf357700 1 -- 172.21.15.43:0/195666706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db80cd9c0 con 0x7f8db80cb5c0 2023-06-28T20:07:36.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.056+0000 7f8dadffb700 1 -- 172.21.15.43:0/195666706 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db000bea0 con 0x7f8db80cb5c0 2023-06-28T20:07:36.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.057+0000 7f8dbf357700 1 -- 172.21.15.43:0/195666706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db80cdf80 con 0x7f8db80cb5c0 2023-06-28T20:07:36.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.058+0000 7f8dbf357700 1 -- 172.21.15.43:0/195666706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db80728b0 con 0x7f8db80cb5c0 2023-06-28T20:07:36.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.058+0000 7f8dadffb700 1 -- 172.21.15.43:0/195666706 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8db0014620 con 0x7f8db80cb5c0 2023-06-28T20:07:36.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.058+0000 7f8dadffb700 1 --2- 172.21.15.43:0/195666706 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f8d9c06ace0 0x7f8d9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.059+0000 7f8dbd0f3700 1 --2- 172.21.15.43:0/195666706 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f8d9c06ace0 0x7f8d9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.059+0000 7f8dadffb700 1 -- 172.21.15.43:0/195666706 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8db00898c0 con 0x7f8db80cb5c0 2023-06-28T20:07:36.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.059+0000 7f8dbd0f3700 1 --2- 172.21.15.43:0/195666706 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f8d9c06ace0 0x7f8d9c06d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8da8009be0 tx=0x7f8da8009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:36.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.063+0000 7f8dadffb700 1 -- 172.21.15.43:0/195666706 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8db0054970 con 0x7f8db80cb5c0 2023-06-28T20:07:36.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.307+0000 7f8dbf357700 1 -- 172.21.15.43:0/195666706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8db80c6250 con 0x7f8db80cb5c0 2023-06-28T20:07:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.308+0000 7f8dadffb700 1 -- 172.21.15.43:0/195666706 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4665 (secure 0 0 0) 0x7f8db0057f90 con 0x7f8db80cb5c0 2023-06-28T20:07:36.312 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:36.312 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4033115951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4033115951},{"type":"v1","addr":"172.21.15.178:6835","nonce":4033115951}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/65569869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":65569869},{"type":"v1","addr":"172.21.15.43:6837","nonce":65569869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/1016213057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016213057},{"type":"v1","addr":"172.21.15.178:6834","nonce":1016213057}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-28T20:07:20.788417+0000","modified":"2023-06-28T20:07:24.388714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/3514544484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3514544484},{"type":"v1","addr":"172.21.15.43:6835","nonce":3514544484}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-28T20:07:36.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.310+0000 7f8dbf357700 1 -- 172.21.15.43:0/195666706 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f8d9c06ace0 msgr2=0x7f8d9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:36.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.310+0000 7f8dbf357700 1 --2- 172.21.15.43:0/195666706 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f8d9c06ace0 0x7f8d9c06d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8da8009be0 tx=0x7f8da8009210 comp rx=0 tx=0).stop 2023-06-28T20:07:36.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.310+0000 7f8dbf357700 1 -- 172.21.15.43:0/195666706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8db80cb5c0 msgr2=0x7f8db80a9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:36.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.310+0000 7f8dbf357700 1 --2- 172.21.15.43:0/195666706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8db80cb5c0 0x7f8db80a9060 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8db000eed0 tx=0x7f8db000b390 comp rx=0 tx=0).stop 2023-06-28T20:07:36.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.310+0000 7f8dbf357700 1 -- 172.21.15.43:0/195666706 shutdown_connections 2023-06-28T20:07:36.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.310+0000 7f8dbf357700 1 --2- 172.21.15.43:0/195666706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8db80cb5c0 0x7f8db80a9060 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.310+0000 7f8dbf357700 1 --2- 172.21.15.43:0/195666706 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f8d9c06ace0 0x7f8d9c06d190 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.310+0000 7f8dbf357700 1 --2- 172.21.15.43:0/195666706 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8db806b230 0x7f8db80a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.310+0000 7f8dbf357700 1 --2- 172.21.15.43:0/195666706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8db80cb1f0 0x7f8db81f22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.310+0000 7f8dbf357700 1 -- 172.21.15.43:0/195666706 >> 172.21.15.43:0/195666706 conn(0x7f8db806c170 msgr2=0x7f8db80c4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:36.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.310+0000 7f8dbf357700 1 -- 172.21.15.43:0/195666706 shutdown_connections 2023-06-28T20:07:36.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.310+0000 7f8dbf357700 1 -- 172.21.15.43:0/195666706 wait complete. 2023-06-28T20:07:36.318 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 7 2023-06-28T20:07:36.728 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-06-28T20:07:20.788417+0000', 'modified': '2023-06-28T20:07:24.388714+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4300}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4300': {'gid': 4300, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.43:6835/3514544484', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6834', 'nonce': 3514544484}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 3514544484}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-06-28T20:07:36.729 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-06-28T20:07:36.729 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-06-28T20:07:36.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.856+0000 7f591beca700 1 Processor -- start 2023-06-28T20:07:36.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.856+0000 7f591beca700 1 -- start start 2023-06-28T20:07:36.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.857+0000 7f591beca700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f59140af060 0x7f5914160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.857+0000 7f591beca700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f591406ccc0 0x7f59140b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.857+0000 7f591beca700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5914160250 con 0x7f59140aec90 2023-06-28T20:07:36.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.857+0000 7f591beca700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591415cb30 con 0x7f59140af060 2023-06-28T20:07:36.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.857+0000 7f591beca700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5914071410 con 0x7f591406ccc0 2023-06-28T20:07:36.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.857+0000 7f5919465700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f59140af060 0x7f5914160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.857+0000 7f5919c66700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f59140aec90 0x7f5914070bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42806/0 (socket says 172.21.15.43:42806) 2023-06-28T20:07:36.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.857+0000 7f5919465700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f59140af060 0x7f5914160cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53622/0 (socket says 172.21.15.43:53622) 2023-06-28T20:07:36.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.857+0000 7f5919c66700 1 -- 172.21.15.43:0/3604026910 learned_addr learned my addr 172.21.15.43:0/3604026910 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:36.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.857+0000 7f5919465700 1 -- 172.21.15.43:0/3604026910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f591406ccc0 msgr2=0x7f59140b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:36.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.857+0000 7f5919465700 1 --2- 172.21.15.43:0/3604026910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f591406ccc0 0x7f59140b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.857+0000 7f5919465700 1 -- 172.21.15.43:0/3604026910 >> v1:172.21.15.43:6789/0 conn(0x7f59140aec90 legacy=0x7f5914070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:36.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.857+0000 7f5919465700 1 -- 172.21.15.43:0/3604026910 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59141604f0 con 0x7f59140af060 2023-06-28T20:07:36.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.857+0000 7f5918c64700 1 -- 172.21.15.43:0/3604026910 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 50534780 0 0) 0x7f5914160250 con 0x7f59140aec90 2023-06-28T20:07:36.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.858+0000 7f5919465700 1 --2- 172.21.15.43:0/3604026910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59140af060 0x7f5914160cf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5910009840 tx=0x7f5910009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606758ffc06c237c server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:36.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.858+0000 7f5918c64700 1 -- 172.21.15.43:0/3604026910 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5910011830 con 0x7f59140af060 2023-06-28T20:07:36.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.858+0000 7f5918c64700 1 -- 172.21.15.43:0/3604026910 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5910011990 con 0x7f59140af060 2023-06-28T20:07:36.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.858+0000 7f5918c64700 1 -- 172.21.15.43:0/3604026910 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5910011d20 con 0x7f59140af060 2023-06-28T20:07:36.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.858+0000 7f591beca700 1 -- 172.21.15.43:0/3604026910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59140af060 msgr2=0x7f5914160cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:36.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.858+0000 7f591beca700 1 --2- 172.21.15.43:0/3604026910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59140af060 0x7f5914160cf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5910009840 tx=0x7f5910009b50 comp rx=0 tx=0).stop 2023-06-28T20:07:36.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.858+0000 7f591beca700 1 -- 172.21.15.43:0/3604026910 shutdown_connections 2023-06-28T20:07:36.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.858+0000 7f591beca700 1 --2- 172.21.15.43:0/3604026910 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f591406ccc0 0x7f59140b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.858+0000 7f591beca700 1 --2- 172.21.15.43:0/3604026910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59140af060 0x7f5914160cf0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.858+0000 7f591beca700 1 -- 172.21.15.43:0/3604026910 >> 172.21.15.43:0/3604026910 conn(0x7f591406b000 msgr2=0x7f591406b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:36.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.858+0000 7f591beca700 1 -- 172.21.15.43:0/3604026910 shutdown_connections 2023-06-28T20:07:36.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.858+0000 7f591beca700 1 -- 172.21.15.43:0/3604026910 wait complete. 2023-06-28T20:07:36.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.860+0000 7f591beca700 1 Processor -- start 2023-06-28T20:07:36.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.860+0000 7f591beca700 1 -- start start 2023-06-28T20:07:36.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.860+0000 7f591beca700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f591406ccc0 0x7f591415ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.860+0000 7f591beca700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59140aec90 0x7f59140b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.860+0000 7f591beca700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59140af060 0x7f59141f5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.860+0000 7f591beca700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59140b4b90 con 0x7f59140af060 2023-06-28T20:07:36.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.860+0000 7f591beca700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59141f5ec0 con 0x7f59140aec90 2023-06-28T20:07:36.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.860+0000 7f591beca700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59141f61a0 con 0x7f591406ccc0 2023-06-28T20:07:36.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.861+0000 7f591a467700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59140af060 0x7f59141f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.861+0000 7f591a467700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59140af060 0x7f59141f5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53630/0 (socket says 172.21.15.43:53630) 2023-06-28T20:07:36.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.861+0000 7f591a467700 1 -- 172.21.15.43:0/2443146477 learned_addr learned my addr 172.21.15.43:0/2443146477 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:36.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.861+0000 7f5919465700 1 --2- 172.21.15.43:0/2443146477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59140aec90 0x7f59140b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.861+0000 7f5919c66700 1 --2- 172.21.15.43:0/2443146477 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f591406ccc0 0x7f591415ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.861+0000 7f591a467700 1 -- 172.21.15.43:0/2443146477 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f591406ccc0 msgr2=0x7f591415ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:36.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.861+0000 7f591a467700 1 --2- 172.21.15.43:0/2443146477 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f591406ccc0 0x7f591415ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.861+0000 7f591a467700 1 -- 172.21.15.43:0/2443146477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59140aec90 msgr2=0x7f59140b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:36.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.861+0000 7f591a467700 1 --2- 172.21.15.43:0/2443146477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59140aec90 0x7f59140b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.861+0000 7f591a467700 1 -- 172.21.15.43:0/2443146477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59100094a0 con 0x7f59140af060 2023-06-28T20:07:36.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.861+0000 7f5919465700 1 --2- 172.21.15.43:0/2443146477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59140aec90 0x7f59140b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:36.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.861+0000 7f591a467700 1 --2- 172.21.15.43:0/2443146477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59140af060 0x7f59141f5690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f590c00b970 tx=0x7f590c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:36.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.861+0000 7f590a7fc700 1 -- 172.21.15.43:0/2443146477 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c00c9b0 con 0x7f59140af060 2023-06-28T20:07:36.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.861+0000 7f590a7fc700 1 -- 172.21.15.43:0/2443146477 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590c00cb30 con 0x7f59140af060 2023-06-28T20:07:36.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.861+0000 7f591beca700 1 -- 172.21.15.43:0/2443146477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59141f6480 con 0x7f59140af060 2023-06-28T20:07:36.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.861+0000 7f590a7fc700 1 -- 172.21.15.43:0/2443146477 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c014440 con 0x7f59140af060 2023-06-28T20:07:36.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.861+0000 7f591beca700 1 -- 172.21.15.43:0/2443146477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59140b2c20 con 0x7f59140af060 2023-06-28T20:07:36.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.863+0000 7f590a7fc700 1 -- 172.21.15.43:0/2443146477 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f590c01d3b0 con 0x7f59140af060 2023-06-28T20:07:36.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.863+0000 7f590a7fc700 1 --2- 172.21.15.43:0/2443146477 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f58f806ace0 0x7f58f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.863+0000 7f5919c66700 1 --2- 172.21.15.43:0/2443146477 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f58f806ace0 0x7f58f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.863+0000 7f590a7fc700 1 -- 172.21.15.43:0/2443146477 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f590c006e40 con 0x7f59140af060 2023-06-28T20:07:36.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.864+0000 7f5919c66700 1 --2- 172.21.15.43:0/2443146477 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f58f806ace0 0x7f58f806d190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5904000f60 tx=0x7f5904007be0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:36.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.865+0000 7f592804eb80 1 Processor -- start 2023-06-28T20:07:36.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.865+0000 7f592804eb80 1 -- start start 2023-06-28T20:07:36.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.866+0000 7f592804eb80 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f63c170 0x55db1f663080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.866+0000 7f592804eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 0x55db1f646f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.866+0000 7f592804eb80 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55db1f6176c0 0x55db1f647470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.882 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.866+0000 7f592804eb80 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db1f521c60 con 0x55db1f63c170 2023-06-28T20:07:36.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.866+0000 7f592804eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db1f5749f0 con 0x55db1f63c540 2023-06-28T20:07:36.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.866+0000 7f592804eb80 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db1f56de80 con 0x55db1f6176c0 2023-06-28T20:07:36.883 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.866+0000 7f5919c66700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f63c170 0x55db1f663080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.866+0000 7f5919c66700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f63c170 0x55db1f663080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53634/0 (socket says 172.21.15.43:53634) 2023-06-28T20:07:36.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.866+0000 7f5919c66700 1 -- 172.21.15.43:0/2538481109 learned_addr learned my addr 172.21.15.43:0/2538481109 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:36.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.866+0000 7f5919465700 1 --2- 172.21.15.43:0/2538481109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 0x55db1f646f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.866+0000 7f591a467700 1 --2- 172.21.15.43:0/2538481109 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55db1f6176c0 0x55db1f647470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.866+0000 7f5919c66700 1 -- 172.21.15.43:0/2538481109 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55db1f6176c0 msgr2=0x55db1f647470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:36.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.866+0000 7f5919c66700 1 --2- 172.21.15.43:0/2538481109 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55db1f6176c0 0x55db1f647470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.884 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.866+0000 7f5919c66700 1 -- 172.21.15.43:0/2538481109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 msgr2=0x55db1f646f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:36.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.866+0000 7f5919c66700 1 --2- 172.21.15.43:0/2538481109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 0x55db1f646f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.867+0000 7f5919c66700 1 -- 172.21.15.43:0/2538481109 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db1f39bd90 con 0x55db1f63c170 2023-06-28T20:07:36.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.867+0000 7f5919465700 1 --2- 172.21.15.43:0/2538481109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 0x55db1f646f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:36.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.867+0000 7f5919c66700 1 --2- 172.21.15.43:0/2538481109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f63c170 0x55db1f663080 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5904015680 tx=0x7f5904012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ded4571716cea server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:36.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.867+0000 7f58f77fe700 1 -- 172.21.15.43:0/2538481109 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904015320 con 0x55db1f63c170 2023-06-28T20:07:36.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.867+0000 7f58f77fe700 1 -- 172.21.15.43:0/2538481109 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5904021450 con 0x55db1f63c170 2023-06-28T20:07:36.885 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.867+0000 7f58f77fe700 1 -- 172.21.15.43:0/2538481109 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59040217c0 con 0x55db1f63c170 2023-06-28T20:07:36.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.867+0000 7f592804eb80 1 -- 172.21.15.43:0/2538481109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f63c170 msgr2=0x55db1f663080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:36.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.867+0000 7f592804eb80 1 --2- 172.21.15.43:0/2538481109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f63c170 0x55db1f663080 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5904015680 tx=0x7f5904012000 comp rx=0 tx=0).stop 2023-06-28T20:07:36.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.867+0000 7f592804eb80 1 -- 172.21.15.43:0/2538481109 shutdown_connections 2023-06-28T20:07:36.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.867+0000 7f592804eb80 1 --2- 172.21.15.43:0/2538481109 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55db1f6176c0 0x55db1f647470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.886 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.867+0000 7f592804eb80 1 --2- 172.21.15.43:0/2538481109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f63c170 0x55db1f663080 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.867+0000 7f592804eb80 1 --2- 172.21.15.43:0/2538481109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 0x55db1f646f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.867+0000 7f592804eb80 1 -- 172.21.15.43:0/2538481109 >> 172.21.15.43:0/2538481109 conn(0x55db1f611a30 msgr2=0x55db1f616fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.868+0000 7f592804eb80 1 -- 172.21.15.43:0/2538481109 shutdown_connections 2023-06-28T20:07:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.868+0000 7f592804eb80 1 -- 172.21.15.43:0/2538481109 wait complete. 2023-06-28T20:07:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.868+0000 7f592804eb80 1 -- start start 2023-06-28T20:07:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.868+0000 7f592804eb80 1 Processor -- start 2023-06-28T20:07:36.887 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.869+0000 7f592804eb80 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55db1f6176c0 0x55db1f63f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.869+0000 7f592804eb80 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f63c170 0x55db1f65ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.869+0000 7f592804eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 0x55db1f619380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.869+0000 7f592804eb80 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db1f50da70 con 0x55db1f63c170 2023-06-28T20:07:36.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.869+0000 7f592804eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db1f50d770 con 0x55db1f63c540 2023-06-28T20:07:36.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.869+0000 7f592804eb80 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db1f3f18d0 con 0x55db1f6176c0 2023-06-28T20:07:36.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.869+0000 7f592804eb80 10 client.0 _ll_register_callbacks cb 0x55db1f50b710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-28T20:07:36.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.869+0000 7f5919465700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f63c170 0x55db1f65ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.869+0000 7f5919465700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f63c170 0x55db1f65ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53642/0 (socket says 172.21.15.43:53642) 2023-06-28T20:07:36.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.869+0000 7f5919465700 1 -- 172.21.15.43:0/2009054614 learned_addr learned my addr 172.21.15.43:0/2009054614 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:36.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.869+0000 7f5919c66700 1 --2- 172.21.15.43:0/2009054614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55db1f6176c0 0x55db1f63f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.869+0000 7f591a467700 1 --2- 172.21.15.43:0/2009054614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 0x55db1f619380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.869+0000 7f5919465700 1 -- 172.21.15.43:0/2009054614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55db1f6176c0 msgr2=0x55db1f63f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:36.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.870+0000 7f5919465700 1 --2- 172.21.15.43:0/2009054614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55db1f6176c0 0x55db1f63f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.870+0000 7f5919465700 1 -- 172.21.15.43:0/2009054614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 msgr2=0x55db1f619380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:36.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.870+0000 7f5919465700 1 --2- 172.21.15.43:0/2009054614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 0x55db1f619380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.870+0000 7f5919465700 1 -- 172.21.15.43:0/2009054614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590400f040 con 0x55db1f63c170 2023-06-28T20:07:36.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.870+0000 7f591a467700 1 --2- 172.21.15.43:0/2009054614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 0x55db1f619380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:36.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.870+0000 7f5919465700 1 --2- 172.21.15.43:0/2009054614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f63c170 0x55db1f65ba30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f59100090b0 tx=0x7f591000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37e11db4597b43 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:36.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.870+0000 7f58f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-06-28T20:07:36.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.870+0000 7f58f4ff9700 1 -- 172.21.15.43:0/2009054614 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5910004510 con 0x55db1f63c170 2023-06-28T20:07:36.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.870+0000 7f592804eb80 1 -- 172.21.15.43:0/2009054614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db1f60f040 con 0x55db1f63c170 2023-06-28T20:07:36.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.870+0000 7f58f4ff9700 1 -- 172.21.15.43:0/2009054614 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f591001f070 con 0x55db1f63c170 2023-06-28T20:07:36.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.870+0000 7f58f4ff9700 1 -- 172.21.15.43:0/2009054614 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59100036a0 con 0x55db1f63c170 2023-06-28T20:07:36.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.870+0000 7f592804eb80 10 client.4665 fetch_fsmap learned FSMap version 7 2023-06-28T20:07:36.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.870+0000 7f592804eb80 1 -- 172.21.15.43:0/2009054614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55db1f384c10 con 0x55db1f63c170 2023-06-28T20:07:36.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.870+0000 7f58f4ff9700 1 -- 172.21.15.43:0/2009054614 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55db1f60f040 con 0x55db1f63c170 2023-06-28T20:07:36.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.870+0000 7f58f4ff9700 1 -- 172.21.15.43:0/2009054614 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 7) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5910003800 con 0x55db1f63c170 2023-06-28T20:07:36.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.870+0000 7f592804eb80 10 client.4665 fetch_fsmap finished waiting for FSMap version 7 2023-06-28T20:07:36.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.870+0000 7f592804eb80 10 client.4665 resolve_mds: resolved name 'c' to daemon mds.c 2023-06-28T20:07:36.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.870+0000 7f592804eb80 20 client.4665 populate_metadata read hostname 'smithi043' 2023-06-28T20:07:36.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.870+0000 7f592804eb80 1 --2- 172.21.15.43:0/2009054614 >> [v2:172.21.15.43:6834/3514544484,v1:172.21.15.43:6835/3514544484] conn(0x55db1f64a670 0x55db1f659130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.870+0000 7f592804eb80 4 client.4665 mds_command: new command op to 4300 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-28T20:07:36.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.870+0000 7f592804eb80 1 -- 172.21.15.43:0/2009054614 --> [v2:172.21.15.43:6834/3514544484,v1:172.21.15.43:6835/3514544484] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db1f566e60 con 0x55db1f64a670 2023-06-28T20:07:36.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.870+0000 7f5919c66700 1 --2- 172.21.15.43:0/2009054614 >> [v2:172.21.15.43:6834/3514544484,v1:172.21.15.43:6835/3514544484] conn(0x55db1f64a670 0x55db1f659130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.871+0000 7f58f4ff9700 10 client.4665 ms_handle_connect on v2:172.21.15.43:6834/3514544484 2023-06-28T20:07:36.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.871+0000 7f5919c66700 1 --2- 172.21.15.43:0/2009054614 >> [v2:172.21.15.43:6834/3514544484,v1:172.21.15.43:6835/3514544484] conn(0x55db1f64a670 0x55db1f659130 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23924506219aad85 server_cookie=d18a73ba6b414ad6 in_seq=0 out_seq=0 2023-06-28T20:07:36.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.871+0000 7f58f4ff9700 1 -- 172.21.15.43:0/2009054614 <== mds.0 v2:172.21.15.43:6834/3514544484 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55db1f566e60 con 0x55db1f64a670 2023-06-28T20:07:36.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.871+0000 7f58f4ff9700 10 client.4665 handle_command_reply: tid=0 2023-06-28T20:07:36.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.871+0000 7f592804eb80 1 client.4665 shutdown 2023-06-28T20:07:36.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.872+0000 7f592804eb80 1 -- 172.21.15.43:0/2009054614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f63c170 msgr2=0x55db1f65ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:36.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.872+0000 7f592804eb80 1 --2- 172.21.15.43:0/2009054614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f63c170 0x55db1f65ba30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f59100090b0 tx=0x7f591000e000 comp rx=0 tx=0).stop 2023-06-28T20:07:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.872+0000 7f592804eb80 1 -- 172.21.15.43:0/2009054614 shutdown_connections 2023-06-28T20:07:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.872+0000 7f592804eb80 1 --2- 172.21.15.43:0/2009054614 >> [v2:172.21.15.43:6834/3514544484,v1:172.21.15.43:6835/3514544484] conn(0x55db1f64a670 0x55db1f659130 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.872+0000 7f592804eb80 1 --2- 172.21.15.43:0/2009054614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 0x55db1f619380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.872+0000 7f592804eb80 1 --2- 172.21.15.43:0/2009054614 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55db1f6176c0 0x55db1f63f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.872+0000 7f58f4ff9700 0 client.4665 ms_handle_reset on v2:172.21.15.43:6834/3514544484 2023-06-28T20:07:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.872+0000 7f592804eb80 1 --2- 172.21.15.43:0/2009054614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f63c170 0x55db1f65ba30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.872+0000 7f592804eb80 1 -- 172.21.15.43:0/2009054614 >> 172.21.15.43:0/2009054614 conn(0x55db1f611a30 msgr2=0x55db1f6653c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.872+0000 7f592804eb80 1 -- 172.21.15.43:0/2009054614 shutdown_connections 2023-06-28T20:07:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.872+0000 7f592804eb80 1 -- 172.21.15.43:0/2009054614 wait complete. 2023-06-28T20:07:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.872+0000 7f592804eb80 20 client.4665 trim_cache size 0 max 16384 2023-06-28T20:07:36.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.890+0000 7f592804eb80 1 Processor -- start 2023-06-28T20:07:36.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.890+0000 7f592804eb80 1 -- start start 2023-06-28T20:07:36.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.890+0000 7f592804eb80 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f6176c0 0x55db1f617e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.890+0000 7f592804eb80 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55db1f63c170 0x55db1f6482e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.890+0000 7f592804eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 0x55db1f648820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.890+0000 7f592804eb80 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db1f61afa0 con 0x55db1f6176c0 2023-06-28T20:07:36.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.890+0000 7f592804eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db1f611840 con 0x55db1f63c540 2023-06-28T20:07:36.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.890+0000 7f592804eb80 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db1f65a6a0 con 0x55db1f63c170 2023-06-28T20:07:36.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.890+0000 7f5919c66700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f6176c0 0x55db1f617e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.890+0000 7f5919c66700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f6176c0 0x55db1f617e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53648/0 (socket says 172.21.15.43:53648) 2023-06-28T20:07:36.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.890+0000 7f5919c66700 1 -- 172.21.15.43:0/3403349872 learned_addr learned my addr 172.21.15.43:0/3403349872 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:36.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.891+0000 7f5919465700 1 --2- 172.21.15.43:0/3403349872 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55db1f63c170 0x55db1f6482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.891+0000 7f591a467700 1 --2- 172.21.15.43:0/3403349872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 0x55db1f648820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.891+0000 7f5919c66700 1 -- 172.21.15.43:0/3403349872 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55db1f63c170 msgr2=0x55db1f6482e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:36.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.891+0000 7f5919c66700 1 --2- 172.21.15.43:0/3403349872 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55db1f63c170 0x55db1f6482e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.891+0000 7f5919c66700 1 -- 172.21.15.43:0/3403349872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 msgr2=0x55db1f648820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:36.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.891+0000 7f5919c66700 1 --2- 172.21.15.43:0/3403349872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 0x55db1f648820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.891+0000 7f5919c66700 1 -- 172.21.15.43:0/3403349872 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db1f3c2200 con 0x55db1f6176c0 2023-06-28T20:07:36.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.891+0000 7f5919465700 1 --2- 172.21.15.43:0/3403349872 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55db1f63c170 0x55db1f6482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:36.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.891+0000 7f591a467700 1 --2- 172.21.15.43:0/3403349872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 0x55db1f648820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:36.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.891+0000 7f5919c66700 1 --2- 172.21.15.43:0/3403349872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f6176c0 0x55db1f617e40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5904015650 tx=0x7f590400a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301ca3948ffe436 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:36.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.891+0000 7f58f7fff700 1 -- 172.21.15.43:0/3403349872 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904028040 con 0x55db1f6176c0 2023-06-28T20:07:36.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.891+0000 7f58f7fff700 1 -- 172.21.15.43:0/3403349872 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590400b030 con 0x55db1f6176c0 2023-06-28T20:07:36.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.891+0000 7f58f7fff700 1 -- 172.21.15.43:0/3403349872 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590402e620 con 0x55db1f6176c0 2023-06-28T20:07:36.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.891+0000 7f592804eb80 1 -- 172.21.15.43:0/3403349872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f6176c0 msgr2=0x55db1f617e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:36.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.891+0000 7f592804eb80 1 --2- 172.21.15.43:0/3403349872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f6176c0 0x55db1f617e40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5904015650 tx=0x7f590400a000 comp rx=0 tx=0).stop 2023-06-28T20:07:36.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.892+0000 7f592804eb80 1 -- 172.21.15.43:0/3403349872 shutdown_connections 2023-06-28T20:07:36.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.892+0000 7f592804eb80 1 --2- 172.21.15.43:0/3403349872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 0x55db1f648820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.892+0000 7f592804eb80 1 --2- 172.21.15.43:0/3403349872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f6176c0 0x55db1f617e40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.892+0000 7f592804eb80 1 --2- 172.21.15.43:0/3403349872 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55db1f63c170 0x55db1f6482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.892+0000 7f592804eb80 1 -- 172.21.15.43:0/3403349872 >> 172.21.15.43:0/3403349872 conn(0x55db1f611a30 msgr2=0x55db1f683d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:36.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.892+0000 7f592804eb80 1 -- 172.21.15.43:0/3403349872 shutdown_connections 2023-06-28T20:07:36.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.892+0000 7f592804eb80 1 -- 172.21.15.43:0/3403349872 wait complete. 2023-06-28T20:07:36.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.892+0000 7f592804eb80 1 -- start start 2023-06-28T20:07:36.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.892+0000 7f592804eb80 1 Processor -- start 2023-06-28T20:07:36.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.893+0000 7f592804eb80 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f6176c0 0x55db1f65ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.893+0000 7f592804eb80 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55db1f63c170 0x55db1f6c8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.893+0000 7f592804eb80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 0x55db1f615cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.893+0000 7f592804eb80 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db1f663950 con 0x55db1f6176c0 2023-06-28T20:07:36.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.893+0000 7f592804eb80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db1f6596f0 con 0x55db1f63c540 2023-06-28T20:07:36.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.893+0000 7f592804eb80 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db1f61b1d0 con 0x55db1f63c170 2023-06-28T20:07:36.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.893+0000 7f592804eb80 10 client.0 _ll_register_callbacks cb 0x7f59100096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-28T20:07:36.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.893+0000 7f5919c66700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f6176c0 0x55db1f65ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.893+0000 7f5919c66700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f6176c0 0x55db1f65ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53654/0 (socket says 172.21.15.43:53654) 2023-06-28T20:07:36.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.893+0000 7f5919c66700 1 -- 172.21.15.43:0/2761122466 learned_addr learned my addr 172.21.15.43:0/2761122466 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:36.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.893+0000 7f5919465700 1 --2- 172.21.15.43:0/2761122466 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55db1f63c170 0x55db1f6c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.893+0000 7f591a467700 1 --2- 172.21.15.43:0/2761122466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 0x55db1f615cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.893+0000 7f5919c66700 1 -- 172.21.15.43:0/2761122466 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55db1f63c170 msgr2=0x55db1f6c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:36.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.893+0000 7f5919c66700 1 --2- 172.21.15.43:0/2761122466 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55db1f63c170 0x55db1f6c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.893+0000 7f5919c66700 1 -- 172.21.15.43:0/2761122466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 msgr2=0x55db1f615cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:36.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.893+0000 7f5919c66700 1 --2- 172.21.15.43:0/2761122466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 0x55db1f615cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.893+0000 7f5919c66700 1 -- 172.21.15.43:0/2761122466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db1f39bd90 con 0x55db1f6176c0 2023-06-28T20:07:36.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.893+0000 7f591a467700 1 --2- 172.21.15.43:0/2761122466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 0x55db1f615cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:36.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.894+0000 7f5919c66700 1 --2- 172.21.15.43:0/2761122466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f6176c0 0x55db1f65ad80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5904017010 tx=0x7f5904000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279c816089727353 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:36.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.894+0000 7f58f4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-06-28T20:07:36.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.894+0000 7f58f4ff9700 1 -- 172.21.15.43:0/2761122466 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590402e5e0 con 0x55db1f6176c0 2023-06-28T20:07:36.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.894+0000 7f58f4ff9700 1 -- 172.21.15.43:0/2761122466 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f590402ebd0 con 0x55db1f6176c0 2023-06-28T20:07:36.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.894+0000 7f592804eb80 1 -- 172.21.15.43:0/2761122466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5904021450 con 0x55db1f6176c0 2023-06-28T20:07:36.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.894+0000 7f58f4ff9700 1 -- 172.21.15.43:0/2761122466 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f590402ed90 con 0x55db1f6176c0 2023-06-28T20:07:36.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.894+0000 7f592804eb80 10 client.4671 fetch_fsmap learned FSMap version 7 2023-06-28T20:07:36.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.894+0000 7f592804eb80 1 -- 172.21.15.43:0/2761122466 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55db1f39a260 con 0x55db1f6176c0 2023-06-28T20:07:36.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.894+0000 7f58f4ff9700 1 -- 172.21.15.43:0/2761122466 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5904021450 con 0x55db1f6176c0 2023-06-28T20:07:36.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.894+0000 7f58f4ff9700 1 -- 172.21.15.43:0/2761122466 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 7) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5904022840 con 0x55db1f6176c0 2023-06-28T20:07:36.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.894+0000 7f592804eb80 10 client.4671 fetch_fsmap finished waiting for FSMap version 7 2023-06-28T20:07:36.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.894+0000 7f592804eb80 10 client.4671 resolve_mds: resolved name 'c' to daemon mds.c 2023-06-28T20:07:36.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.894+0000 7f592804eb80 20 client.4671 populate_metadata read hostname 'smithi043' 2023-06-28T20:07:36.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.894+0000 7f592804eb80 1 --2- 172.21.15.43:0/2761122466 >> [v2:172.21.15.43:6834/3514544484,v1:172.21.15.43:6835/3514544484] conn(0x55db1f6163f0 0x55db1f661920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:36.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.894+0000 7f592804eb80 4 client.4671 mds_command: new command op to 4300 tid=0[{"prefix": "status"}] 2023-06-28T20:07:36.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.894+0000 7f592804eb80 1 -- 172.21.15.43:0/2761122466 --> [v2:172.21.15.43:6834/3514544484,v1:172.21.15.43:6835/3514544484] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55db1f529000 con 0x55db1f6163f0 2023-06-28T20:07:36.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.894+0000 7f5919465700 1 --2- 172.21.15.43:0/2761122466 >> [v2:172.21.15.43:6834/3514544484,v1:172.21.15.43:6835/3514544484] conn(0x55db1f6163f0 0x55db1f661920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:36.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.895+0000 7f58f4ff9700 10 client.4671 ms_handle_connect on v2:172.21.15.43:6834/3514544484 2023-06-28T20:07:36.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.895+0000 7f5919465700 1 --2- 172.21.15.43:0/2761122466 >> [v2:172.21.15.43:6834/3514544484,v1:172.21.15.43:6835/3514544484] conn(0x55db1f6163f0 0x55db1f661920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32c679a23da056c server_cookie=e29ef939db8031b6 in_seq=0 out_seq=0 2023-06-28T20:07:36.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.895+0000 7f58f4ff9700 1 -- 172.21.15.43:0/2761122466 <== mds.0 v2:172.21.15.43:6834/3514544484 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55db1f529000 con 0x55db1f6163f0 2023-06-28T20:07:36.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.895+0000 7f58f4ff9700 10 client.4671 handle_command_reply: tid=0 2023-06-28T20:07:36.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.895+0000 7f592804eb80 1 client.4671 shutdown 2023-06-28T20:07:36.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.895+0000 7f592804eb80 1 -- 172.21.15.43:0/2761122466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f6176c0 msgr2=0x55db1f65ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:36.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.895+0000 7f592804eb80 1 --2- 172.21.15.43:0/2761122466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f6176c0 0x55db1f65ad80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5904017010 tx=0x7f5904000ed0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.895+0000 7f592804eb80 1 -- 172.21.15.43:0/2761122466 shutdown_connections 2023-06-28T20:07:36.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.895+0000 7f592804eb80 1 --2- 172.21.15.43:0/2761122466 >> [v2:172.21.15.43:6834/3514544484,v1:172.21.15.43:6835/3514544484] conn(0x55db1f6163f0 0x55db1f661920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.895+0000 7f592804eb80 1 --2- 172.21.15.43:0/2761122466 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55db1f63c540 0x55db1f615cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.895+0000 7f592804eb80 1 --2- 172.21.15.43:0/2761122466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55db1f6176c0 0x55db1f65ad80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.895+0000 7f58f4ff9700 0 client.4671 ms_handle_reset on v2:172.21.15.43:6834/3514544484 2023-06-28T20:07:36.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.895+0000 7f592804eb80 1 --2- 172.21.15.43:0/2761122466 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55db1f63c170 0x55db1f6c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.895+0000 7f592804eb80 1 -- 172.21.15.43:0/2761122466 >> 172.21.15.43:0/2761122466 conn(0x55db1f611a30 msgr2=0x55db1f617e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:36.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.895+0000 7f592804eb80 1 -- 172.21.15.43:0/2761122466 shutdown_connections 2023-06-28T20:07:36.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.895+0000 7f592804eb80 1 -- 172.21.15.43:0/2761122466 wait complete. 2023-06-28T20:07:36.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.895+0000 7f592804eb80 20 client.4671 trim_cache size 0 max 16384 2023-06-28T20:07:36.919 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-06-28T20:07:36.919 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "b8fd740d-00b9-4308-92a8-48f5fffe8078", 2023-06-28T20:07:36.919 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2023-06-28T20:07:36.919 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 4300, 2023-06-28T20:07:36.919 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2023-06-28T20:07:36.919 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2023-06-28T20:07:36.920 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2023-06-28T20:07:36.920 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 15.533304429999999, 2023-06-28T20:07:36.920 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 7, 2023-06-28T20:07:36.920 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 21, 2023-06-28T20:07:36.920 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 21, 2023-06-28T20:07:36.921 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 21.467807203 2023-06-28T20:07:36.921 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-06-28T20:07:36.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.896+0000 7f591beca700 1 -- 172.21.15.43:0/2443146477 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f58f806ace0 msgr2=0x7f58f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:36.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.896+0000 7f591beca700 1 --2- 172.21.15.43:0/2443146477 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f58f806ace0 0x7f58f806d190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5904000f60 tx=0x7f5904007be0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.896+0000 7f591beca700 1 -- 172.21.15.43:0/2443146477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59140af060 msgr2=0x7f59141f5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:36.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.896+0000 7f591beca700 1 --2- 172.21.15.43:0/2443146477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59140af060 0x7f59141f5690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f590c00b970 tx=0x7f590c00bd30 comp rx=0 tx=0).stop 2023-06-28T20:07:36.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.896+0000 7f591beca700 1 -- 172.21.15.43:0/2443146477 shutdown_connections 2023-06-28T20:07:36.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.896+0000 7f591beca700 1 --2- 172.21.15.43:0/2443146477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59140af060 0x7f59141f5690 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.896+0000 7f591beca700 1 --2- 172.21.15.43:0/2443146477 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f58f806ace0 0x7f58f806d190 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.896+0000 7f591beca700 1 --2- 172.21.15.43:0/2443146477 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f591406ccc0 0x7f591415ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.896+0000 7f591beca700 1 --2- 172.21.15.43:0/2443146477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59140aec90 0x7f59140b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:36.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.896+0000 7f591beca700 1 -- 172.21.15.43:0/2443146477 >> 172.21.15.43:0/2443146477 conn(0x7f591406b000 msgr2=0x7f59140cf750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:36.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.896+0000 7f591beca700 1 -- 172.21.15.43:0/2443146477 shutdown_connections 2023-06-28T20:07:36.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:36.896+0000 7f591beca700 1 -- 172.21.15.43:0/2443146477 wait complete. 2023-06-28T20:07:36.925 INFO:teuthology.run_tasks:Running task kclient... 2023-06-28T20:07:36.952 INFO:tasks.kclient:Mounting kernel clients... 2023-06-28T20:07:36.953 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-06-28T20:07:36.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:36.955 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-06-28T20:07:36.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:36.973 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2023-06-28T20:07:37.042 INFO:teuthology.orchestra.run.smithi043.stderr:Cannot find device "ceph-brx" 2023-06-28T20:07:37.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-28T20:07:37.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:37.044 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-06-28T20:07:37.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:37.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2023-06-28T20:07:37.202 INFO:teuthology.orchestra.run.smithi043.stderr:Cannot find device "ceph-brx" 2023-06-28T20:07:37.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-28T20:07:37.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:37.204 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2023-06-28T20:07:37.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:37.225 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link delete ceph-brx 2023-06-28T20:07:37.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-28T20:07:37.301 INFO:teuthology.orchestra.run.smithi178.stderr:Cannot find device "ceph-brx" 2023-06-28T20:07:37.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:37.301 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2023-06-28T20:07:37.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:37.361 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link delete ceph-brx 2023-06-28T20:07:37.431 INFO:teuthology.orchestra.run.smithi178.stderr:Cannot find device "ceph-brx" 2023-06-28T20:07:37.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-28T20:07:37.433 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-06-28T20:07:37.434 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-28T20:07:37.434 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-06-28T20:07:37.435 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-28T20:07:37.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.558+0000 7f2ef0579700 1 Processor -- start 2023-06-28T20:07:37.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.559+0000 7f2ef0579700 1 -- start start 2023-06-28T20:07:37.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.560+0000 7f2ef0579700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2ee80d10f0 0x7f2ee80ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:37.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.560+0000 7f2ef0579700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2ee80cbab0 0x7f2ee80b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:37.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.560+0000 7f2ef0579700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee815ab50 con 0x7f2ee80cedf0 2023-06-28T20:07:37.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.560+0000 7f2ef0579700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee80cbf30 con 0x7f2ee80cbab0 2023-06-28T20:07:37.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.560+0000 7f2ef0579700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee80cf260 con 0x7f2ee80d10f0 2023-06-28T20:07:37.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.560+0000 7f2eeeb16700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2ee80cbab0 0x7f2ee80b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:37.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.560+0000 7f2eeeb16700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2ee80cbab0 0x7f2ee80b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53666/0 (socket says 172.21.15.43:53666) 2023-06-28T20:07:37.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.560+0000 7f2eeeb16700 1 -- 172.21.15.43:0/4209936727 learned_addr learned my addr 172.21.15.43:0/4209936727 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:37.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.560+0000 7f2eee315700 1 --2- 172.21.15.43:0/4209936727 >> v2:172.21.15.178:3300/0 conn(0x7f2ee80d10f0 0x7f2ee80ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:37.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.560+0000 7f2eeeb16700 1 -- 172.21.15.43:0/4209936727 >> v2:172.21.15.178:3300/0 conn(0x7f2ee80d10f0 msgr2=0x7f2ee80ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:37.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.560+0000 7f2eeeb16700 1 --2- 172.21.15.43:0/4209936727 >> v2:172.21.15.178:3300/0 conn(0x7f2ee80d10f0 0x7f2ee80ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:37.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.560+0000 7f2eeeb16700 1 -- 172.21.15.43:0/4209936727 >> v1:172.21.15.178:6789/0 conn(0x7f2ee80cedf0 legacy=0x7f2ee80cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:37.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.560+0000 7f2eeeb16700 1 -- 172.21.15.43:0/4209936727 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee80cd1b0 con 0x7f2ee80cbab0 2023-06-28T20:07:37.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.561+0000 7f2eeeb16700 1 --2- 172.21.15.43:0/4209936727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee80cbab0 0x7f2ee80b0600 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2ee804e3a0 tx=0x7f2ee000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785cbf8cc0d64977 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:37.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.561+0000 7f2eed313700 1 -- 172.21.15.43:0/4209936727 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee000e070 con 0x7f2ee80cbab0 2023-06-28T20:07:37.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.561+0000 7f2eed313700 1 -- 172.21.15.43:0/4209936727 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee000b8d0 con 0x7f2ee80cbab0 2023-06-28T20:07:37.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.561+0000 7f2eed313700 1 -- 172.21.15.43:0/4209936727 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee000bc80 con 0x7f2ee80cbab0 2023-06-28T20:07:37.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.561+0000 7f2ef0579700 1 -- 172.21.15.43:0/4209936727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee80cbab0 msgr2=0x7f2ee80b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:37.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.561+0000 7f2ef0579700 1 --2- 172.21.15.43:0/4209936727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee80cbab0 0x7f2ee80b0600 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2ee804e3a0 tx=0x7f2ee000b330 comp rx=0 tx=0).stop 2023-06-28T20:07:37.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.561+0000 7f2ef0579700 1 -- 172.21.15.43:0/4209936727 shutdown_connections 2023-06-28T20:07:37.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.561+0000 7f2ef0579700 1 --2- 172.21.15.43:0/4209936727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee80cbab0 0x7f2ee80b0600 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:37.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.561+0000 7f2ef0579700 1 --2- 172.21.15.43:0/4209936727 >> v2:172.21.15.178:3300/0 conn(0x7f2ee80d10f0 0x7f2ee80ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:37.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.561+0000 7f2ef0579700 1 -- 172.21.15.43:0/4209936727 >> 172.21.15.43:0/4209936727 conn(0x7f2ee80664b0 msgr2=0x7f2ee809a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:37.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.561+0000 7f2ef0579700 1 -- 172.21.15.43:0/4209936727 shutdown_connections 2023-06-28T20:07:37.568 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.561+0000 7f2ef0579700 1 -- 172.21.15.43:0/4209936727 wait complete. 2023-06-28T20:07:37.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.563+0000 7f2ef0579700 1 Processor -- start 2023-06-28T20:07:37.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.563+0000 7f2ef0579700 1 -- start start 2023-06-28T20:07:37.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.563+0000 7f2ef0579700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ee80cbab0 0x7f2ee806ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:37.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.563+0000 7f2ef0579700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee80cedf0 0x7f2ee80afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:37.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.563+0000 7f2ef0579700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ee80d10f0 0x7f2ee8159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:37.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.563+0000 7f2ef0579700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee80cb610 con 0x7f2ee80cedf0 2023-06-28T20:07:37.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.563+0000 7f2ef0579700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee80ca990 con 0x7f2ee80cbab0 2023-06-28T20:07:37.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.563+0000 7f2ef0579700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee80cac70 con 0x7f2ee80d10f0 2023-06-28T20:07:37.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.563+0000 7f2eedb14700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee80cedf0 0x7f2ee80afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:37.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.563+0000 7f2eedb14700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee80cedf0 0x7f2ee80afdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53670/0 (socket says 172.21.15.43:53670) 2023-06-28T20:07:37.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.563+0000 7f2eedb14700 1 -- 172.21.15.43:0/3728678834 learned_addr learned my addr 172.21.15.43:0/3728678834 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:37.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.563+0000 7f2eee315700 1 --2- 172.21.15.43:0/3728678834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ee80cbab0 0x7f2ee806ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:37.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.564+0000 7f2eeeb16700 1 --2- 172.21.15.43:0/3728678834 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ee80d10f0 0x7f2ee8159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:37.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.564+0000 7f2eedb14700 1 -- 172.21.15.43:0/3728678834 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ee80d10f0 msgr2=0x7f2ee8159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:37.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.564+0000 7f2eedb14700 1 --2- 172.21.15.43:0/3728678834 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ee80d10f0 0x7f2ee8159670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:37.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.564+0000 7f2eedb14700 1 -- 172.21.15.43:0/3728678834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ee80cbab0 msgr2=0x7f2ee806ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:37.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.564+0000 7f2eedb14700 1 --2- 172.21.15.43:0/3728678834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ee80cbab0 0x7f2ee806ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:37.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.564+0000 7f2eedb14700 1 -- 172.21.15.43:0/3728678834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee0009b30 con 0x7f2ee80cedf0 2023-06-28T20:07:37.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.564+0000 7f2eee315700 1 --2- 172.21.15.43:0/3728678834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ee80cbab0 0x7f2ee806ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:37.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.564+0000 7f2eedb14700 1 --2- 172.21.15.43:0/3728678834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee80cedf0 0x7f2ee80afdd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400c490 tx=0x7f2ee400c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:37.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.564+0000 7f2edaffd700 1 -- 172.21.15.43:0/3728678834 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee400cea0 con 0x7f2ee80cedf0 2023-06-28T20:07:37.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.564+0000 7f2edaffd700 1 -- 172.21.15.43:0/3728678834 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee40052f0 con 0x7f2ee80cedf0 2023-06-28T20:07:37.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.564+0000 7f2edaffd700 1 -- 172.21.15.43:0/3728678834 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4006860 con 0x7f2ee80cedf0 2023-06-28T20:07:37.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.564+0000 7f2ef0579700 1 -- 172.21.15.43:0/3728678834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee80caf50 con 0x7f2ee80cedf0 2023-06-28T20:07:37.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.564+0000 7f2ef0579700 1 -- 172.21.15.43:0/3728678834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee806bf50 con 0x7f2ee80cedf0 2023-06-28T20:07:37.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.565+0000 7f2ef0579700 1 -- 172.21.15.43:0/3728678834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee806c110 con 0x7f2ee80cedf0 2023-06-28T20:07:37.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.566+0000 7f2edaffd700 1 -- 172.21.15.43:0/3728678834 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2ee4003680 con 0x7f2ee80cedf0 2023-06-28T20:07:37.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.566+0000 7f2edaffd700 1 --2- 172.21.15.43:0/3728678834 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2ecc06ace0 0x7f2ecc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:37.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.566+0000 7f2edaffd700 1 -- 172.21.15.43:0/3728678834 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2ee4088650 con 0x7f2ee80cedf0 2023-06-28T20:07:37.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.566+0000 7f2eee315700 1 --2- 172.21.15.43:0/3728678834 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2ecc06ace0 0x7f2ecc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:37.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.566+0000 7f2eee315700 1 --2- 172.21.15.43:0/3728678834 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2ecc06ace0 0x7f2ecc06d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009f10 tx=0x7f2edc0091c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:37.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.571+0000 7f2edaffd700 1 -- 172.21.15.43:0/3728678834 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2ee806c110 con 0x7f2ee80cedf0 2023-06-28T20:07:37.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.812+0000 7f2ef0579700 1 -- 172.21.15.43:0/3728678834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2ee806c110 con 0x7f2ee80cedf0 2023-06-28T20:07:37.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.813+0000 7f2edaffd700 1 -- 172.21.15.43:0/3728678834 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2ee806c110 con 0x7f2ee80cedf0 2023-06-28T20:07:37.817 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-06-28T20:07:37.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.815+0000 7f2ef0579700 1 -- 172.21.15.43:0/3728678834 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2ecc06ace0 msgr2=0x7f2ecc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:37.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.815+0000 7f2ef0579700 1 --2- 172.21.15.43:0/3728678834 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2ecc06ace0 0x7f2ecc06d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009f10 tx=0x7f2edc0091c0 comp rx=0 tx=0).stop 2023-06-28T20:07:37.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.815+0000 7f2ef0579700 1 -- 172.21.15.43:0/3728678834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee80cedf0 msgr2=0x7f2ee80afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:37.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.815+0000 7f2ef0579700 1 --2- 172.21.15.43:0/3728678834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee80cedf0 0x7f2ee80afdd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400c490 tx=0x7f2ee400c7a0 comp rx=0 tx=0).stop 2023-06-28T20:07:37.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.815+0000 7f2ef0579700 1 -- 172.21.15.43:0/3728678834 shutdown_connections 2023-06-28T20:07:37.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.815+0000 7f2ef0579700 1 --2- 172.21.15.43:0/3728678834 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2ecc06ace0 0x7f2ecc06d190 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:37.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.815+0000 7f2ef0579700 1 --2- 172.21.15.43:0/3728678834 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2ee80d10f0 0x7f2ee8159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:37.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.815+0000 7f2ef0579700 1 --2- 172.21.15.43:0/3728678834 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ee80cbab0 0x7f2ee806ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:37.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.815+0000 7f2ef0579700 1 --2- 172.21.15.43:0/3728678834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ee80cedf0 0x7f2ee80afdd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:37.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.815+0000 7f2ef0579700 1 -- 172.21.15.43:0/3728678834 >> 172.21.15.43:0/3728678834 conn(0x7f2ee80664b0 msgr2=0x7f2ee80c73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:37.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.815+0000 7f2ef0579700 1 -- 172.21.15.43:0/3728678834 shutdown_connections 2023-06-28T20:07:37.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:37.815+0000 7f2ef0579700 1 -- 172.21.15.43:0/3728678834 wait complete. 2023-06-28T20:07:37.830 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-28T20:07:37.831 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-28T20:07:37.831 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2023-06-28T20:07:37.831 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-06-28T20:07:37.831 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-06-28T20:07:37.832 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-28T20:07:37.832 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-28T20:07:37.832 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-28T20:07:37.832 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-06-28T20:07:37.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:37.833 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2023-06-28T20:07:37.850 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-28T20:07:37.851 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-28T20:07:37.851 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2023-06-28T20:07:37.851 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:37.851 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2023-06-28T20:07:37.851 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:37.852 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-28T20:07:37.852 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2023-06-28T20:07:37.852 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-28T20:07:37.852 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2023-06-28T20:07:37.852 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-28T20:07:37.852 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2023-06-28T20:07:37.852 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-06-28T20:07:37.853 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 41902sec preferred_lft 41902sec 2023-06-28T20:07:37.853 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2023-06-28T20:07:37.853 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:37.853 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-28T20:07:37.853 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2023-06-28T20:07:37.854 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-06-28T20:07:37.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:37.854 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-06-28T20:07:37.854 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-06-28T20:07:37.855 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add name ceph-brx type bridge 2023-06-28T20:07:37.855 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr flush dev ceph-brx 2023-06-28T20:07:37.856 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set ceph-brx up 2023-06-28T20:07:37.856 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-06-28T20:07:37.856 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-06-28T20:07:38.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:38.030 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-28T20:07:38.059 INFO:teuthology.orchestra.run.smithi043.stdout:1 2023-06-28T20:07:38.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:38.061 DEBUG:teuthology.orchestra.run.smithi043:> route 2023-06-28T20:07:38.137 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2023-06-28T20:07:38.137 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-28T20:07:38.137 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-06-28T20:07:38.137 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-06-28T20:07:38.137 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-28T20:07:38.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:38.138 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-06-28T20:07:38.138 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-06-28T20:07:38.139 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-06-28T20:07:38.139 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-06-28T20:07:38.139 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-06-28T20:07:38.139 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-06-28T20:07:38.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:38.420 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-06-28T20:07:38.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:38.441 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list-id 2023-06-28T20:07:38.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:38.498 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-06-28T20:07:38.498 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-06-28T20:07:38.499 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-06-28T20:07:38.499 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-06-28T20:07:38.499 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-06-28T20:07:38.594 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-06-28T20:07:38.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:38.594 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-06-28T20:07:38.595 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-06-28T20:07:38.595 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-06-28T20:07:38.595 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-06-28T20:07:38.596 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-06-28T20:07:38.598 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-06-28T20:07:38.599 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-06-28T20:07:38.599 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-06-28T20:07:38.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:38.815 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-06-28T20:07:38.815 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-06-28T20:07:38.815 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2023-06-28T20:07:38.815 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set dev brx.0 master ceph-brx 2023-06-28T20:07:38.816 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-06-28T20:07:38.919 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-28T20:07:38.919 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-06-28T20:07:38.942 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-06-28T20:07:38.942 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-28T20:07:38.943 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-06-28T20:07:38.999 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-06-28T20:07:39.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:39.000 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-06-28T20:07:39.099 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-06-28T20:07:39.099 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-06-28T20:07:39.099 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2023-06-28T20:07:39.223 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-06-28T20:07:39.223 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-06-28T20:07:39.224 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2023-06-28T20:07:39.224 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.178:3300,172.21.15.178:3301" 2023-06-28T20:07:39.224 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2023-06-28T20:07:39.264 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-06-28T20:07:39.265 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-06-28T20:07:39.265 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2023-06-28T20:07:39.265 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.178:3300,172.21.15.178:3301" 2023-06-28T20:07:39.265 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2023-06-28T20:07:39.265 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.178:6789,172.21.15.178:6790" 2023-06-28T20:07:39.266 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2023-06-28T20:07:39.266 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=b8fd740d-00b9-4308-92a8-48f5fffe8078" will pass to kernel 2023-06-28T20:07:39.273 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-06-28T20:07:39.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:39.273 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-06-28T20:07:39.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:39.308 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-28T20:07:39.391 INFO:teuthology.orchestra.run.smithi043.stdout:client4680 2023-06-28T20:07:39.392 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:07:39.392 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:07:39.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:39.450 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-28T20:07:39.540 INFO:teuthology.orchestra.run.smithi043.stdout:client4680 2023-06-28T20:07:39.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:39.541 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-28T20:07:39.585 INFO:teuthology.orchestra.run.smithi043.stdout:client4680 2023-06-28T20:07:39.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:39.586 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/b8fd740d-00b9-4308-92a8-48f5fffe8078.client4680/status') 2023-06-28T20:07:39.678 INFO:teuthology.orchestra.run.smithi043.stdout:instance: client.4680 (3)192.168.0.1:0/4293618756 2023-06-28T20:07:39.678 INFO:teuthology.orchestra.run.smithi043.stdout:blocklisted: false 2023-06-28T20:07:39.679 INFO:teuthology.orchestra.run.smithi043.stderr:0+1 records in 2023-06-28T20:07:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:0+1 records out 2023-06-28T20:07:39.680 INFO:teuthology.orchestra.run.smithi043.stderr:69 bytes copied, 2.894e-05 s, 2.4 MB/s 2023-06-28T20:07:39.680 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-06-28T20:07:39.680 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-28T20:07:39.681 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-06-28T20:07:39.681 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-28T20:07:39.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.802+0000 7f1fe91bf700 1 Processor -- start 2023-06-28T20:07:39.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.802+0000 7f1fe91bf700 1 -- start start 2023-06-28T20:07:39.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.803+0000 7f1fe91bf700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1fe40d0630 0x7f1fe40d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:39.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.803+0000 7f1fe91bf700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fe40cbd40 0x7f1fe406b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:39.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.803+0000 7f1fe91bf700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1fe406b540 0x7f1fe40cd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.803+0000 7f1fe91bf700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe406b9c0 con 0x7f1fe406b540 2023-06-28T20:07:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.803+0000 7f1fe91bf700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe40ccf10 con 0x7f1fe40d0630 2023-06-28T20:07:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.803+0000 7f1fe91bf700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe415cd80 con 0x7f1fe40cbd40 2023-06-28T20:07:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.803+0000 7f1fe359e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1fe406b540 0x7f1fe40cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.803+0000 7f1fe359e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1fe406b540 0x7f1fe40cd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53682/0 (socket says 172.21.15.43:53682) 2023-06-28T20:07:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.803+0000 7f1fe359e700 1 -- 172.21.15.43:0/3363938480 learned_addr learned my addr 172.21.15.43:0/3363938480 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.804+0000 7f1fe259c700 1 --2- 172.21.15.43:0/3363938480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fe40cbd40 0x7f1fe406b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.804+0000 7f1fe2d9d700 1 --2- 172.21.15.43:0/3363938480 >> v2:172.21.15.178:3300/0 conn(0x7f1fe40d0630 0x7f1fe40d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:39.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.804+0000 7f1fe359e700 1 -- 172.21.15.43:0/3363938480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fe40cbd40 msgr2=0x7f1fe406b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:39.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.804+0000 7f1fe359e700 1 --2- 172.21.15.43:0/3363938480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fe40cbd40 0x7f1fe406b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:39.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.804+0000 7f1fe359e700 1 -- 172.21.15.43:0/3363938480 >> v2:172.21.15.178:3300/0 conn(0x7f1fe40d0630 msgr2=0x7f1fe40d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:39.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.804+0000 7f1fe359e700 1 --2- 172.21.15.43:0/3363938480 >> v2:172.21.15.178:3300/0 conn(0x7f1fe40d0630 0x7f1fe40d1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:39.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.804+0000 7f1fe359e700 1 -- 172.21.15.43:0/3363938480 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe40d0a00 con 0x7f1fe406b540 2023-06-28T20:07:39.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.804+0000 7f1fe359e700 1 --2- 172.21.15.43:0/3363938480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fe406b540 0x7f1fe40cd680 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1fe404e3a0 tx=0x7f1fdc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9447ffd534bc97dc server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:39.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.804+0000 7f1fe1d9b700 1 -- 172.21.15.43:0/3363938480 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc00e070 con 0x7f1fe406b540 2023-06-28T20:07:39.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.804+0000 7f1fe1d9b700 1 -- 172.21.15.43:0/3363938480 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fdc00b9c0 con 0x7f1fe406b540 2023-06-28T20:07:39.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.804+0000 7f1fe1d9b700 1 -- 172.21.15.43:0/3363938480 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc00bd70 con 0x7f1fe406b540 2023-06-28T20:07:39.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.804+0000 7f1fe91bf700 1 -- 172.21.15.43:0/3363938480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fe406b540 msgr2=0x7f1fe40cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:39.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.804+0000 7f1fe91bf700 1 --2- 172.21.15.43:0/3363938480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fe406b540 0x7f1fe40cd680 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1fe404e3a0 tx=0x7f1fdc00b330 comp rx=0 tx=0).stop 2023-06-28T20:07:39.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.805+0000 7f1fe91bf700 1 -- 172.21.15.43:0/3363938480 shutdown_connections 2023-06-28T20:07:39.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.805+0000 7f1fe91bf700 1 --2- 172.21.15.43:0/3363938480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fe406b540 0x7f1fe40cd680 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:39.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.805+0000 7f1fe91bf700 1 --2- 172.21.15.43:0/3363938480 >> v2:172.21.15.178:3300/0 conn(0x7f1fe40d0630 0x7f1fe40d1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:39.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.805+0000 7f1fe91bf700 1 --2- 172.21.15.43:0/3363938480 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fe40cbd40 0x7f1fe406b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:39.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.805+0000 7f1fe91bf700 1 -- 172.21.15.43:0/3363938480 >> 172.21.15.43:0/3363938480 conn(0x7f1fe40664b0 msgr2=0x7f1fe409a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:39.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.805+0000 7f1fe91bf700 1 -- 172.21.15.43:0/3363938480 shutdown_connections 2023-06-28T20:07:39.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.805+0000 7f1fe91bf700 1 -- 172.21.15.43:0/3363938480 wait complete. 2023-06-28T20:07:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.806+0000 7f1fe91bf700 1 Processor -- start 2023-06-28T20:07:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.806+0000 7f1fe91bf700 1 -- start start 2023-06-28T20:07:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.807+0000 7f1fe91bf700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fe406b540 0x7f1fe4152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.807+0000 7f1fe91bf700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fe40cbd40 0x7f1fe415f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.807+0000 7f1fe91bf700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fe40d0630 0x7f1fe40b5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:39.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.807+0000 7f1fe91bf700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe40b3eb0 con 0x7f1fe406b540 2023-06-28T20:07:39.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.807+0000 7f1fe91bf700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe4152740 con 0x7f1fe40cbd40 2023-06-28T20:07:39.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.807+0000 7f1fe91bf700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe40ca790 con 0x7f1fe40d0630 2023-06-28T20:07:39.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.807+0000 7f1fe2d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fe406b540 0x7f1fe4152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:39.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.807+0000 7f1fe2d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fe406b540 0x7f1fe4152200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53684/0 (socket says 172.21.15.43:53684) 2023-06-28T20:07:39.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.807+0000 7f1fe2d9d700 1 -- 172.21.15.43:0/181936966 learned_addr learned my addr 172.21.15.43:0/181936966 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:39.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.807+0000 7f1fe259c700 1 --2- 172.21.15.43:0/181936966 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fe40cbd40 0x7f1fe415f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:39.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.807+0000 7f1fe359e700 1 --2- 172.21.15.43:0/181936966 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fe40d0630 0x7f1fe40b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:39.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.807+0000 7f1fe2d9d700 1 -- 172.21.15.43:0/181936966 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fe40d0630 msgr2=0x7f1fe40b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:39.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.807+0000 7f1fe2d9d700 1 --2- 172.21.15.43:0/181936966 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fe40d0630 0x7f1fe40b5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:39.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.808+0000 7f1fe2d9d700 1 -- 172.21.15.43:0/181936966 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fe40cbd40 msgr2=0x7f1fe415f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:39.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.808+0000 7f1fe2d9d700 1 --2- 172.21.15.43:0/181936966 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fe40cbd40 0x7f1fe415f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:39.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.808+0000 7f1fe2d9d700 1 -- 172.21.15.43:0/181936966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fdc009b30 con 0x7f1fe406b540 2023-06-28T20:07:39.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.808+0000 7f1fe2d9d700 1 --2- 172.21.15.43:0/181936966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fe406b540 0x7f1fe4152200 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1fd400c7f0 tx=0x7f1fd400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:39.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.808+0000 7f1fdb7fe700 1 -- 172.21.15.43:0/181936966 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4011800 con 0x7f1fe406b540 2023-06-28T20:07:39.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.808+0000 7f1fdb7fe700 1 -- 172.21.15.43:0/181936966 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd4011960 con 0x7f1fe406b540 2023-06-28T20:07:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.808+0000 7f1fdb7fe700 1 -- 172.21.15.43:0/181936966 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4011cf0 con 0x7f1fe406b540 2023-06-28T20:07:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.808+0000 7f1fe91bf700 1 -- 172.21.15.43:0/181936966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe40ca1d0 con 0x7f1fe406b540 2023-06-28T20:07:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.808+0000 7f1fe91bf700 1 -- 172.21.15.43:0/181936966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe415d3e0 con 0x7f1fe406b540 2023-06-28T20:07:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.809+0000 7f1fe91bf700 1 -- 172.21.15.43:0/181936966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe40728b0 con 0x7f1fe406b540 2023-06-28T20:07:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.809+0000 7f1fdb7fe700 1 -- 172.21.15.43:0/181936966 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1fd4018360 con 0x7f1fe406b540 2023-06-28T20:07:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.810+0000 7f1fdb7fe700 1 --2- 172.21.15.43:0/181936966 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1fc406ace0 0x7f1fc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.810+0000 7f1fdb7fe700 1 -- 172.21.15.43:0/181936966 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1fd4089350 con 0x7f1fe406b540 2023-06-28T20:07:39.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.810+0000 7f1fe259c700 1 --2- 172.21.15.43:0/181936966 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1fc406ace0 0x7f1fc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:39.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.810+0000 7f1fe259c700 1 --2- 172.21.15.43:0/181936966 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1fc406ace0 0x7f1fc406d190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1fe415c2d0 tx=0x7f1fcc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:39.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:39.814+0000 7f1fdb7fe700 1 -- 172.21.15.43:0/181936966 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1fd4054490 con 0x7f1fe406b540 2023-06-28T20:07:40.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:40.062+0000 7f1fe91bf700 1 -- 172.21.15.43:0/181936966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1fe40cfb60 con 0x7f1fe406b540 2023-06-28T20:07:40.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:40.062+0000 7f1fdb7fe700 1 -- 172.21.15.43:0/181936966 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1fd4057ab0 con 0x7f1fe406b540 2023-06-28T20:07:40.066 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-06-28T20:07:40.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:40.065+0000 7f1fe91bf700 1 -- 172.21.15.43:0/181936966 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1fc406ace0 msgr2=0x7f1fc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:40.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:40.065+0000 7f1fe91bf700 1 --2- 172.21.15.43:0/181936966 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1fc406ace0 0x7f1fc406d190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1fe415c2d0 tx=0x7f1fcc008040 comp rx=0 tx=0).stop 2023-06-28T20:07:40.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:40.065+0000 7f1fe91bf700 1 -- 172.21.15.43:0/181936966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fe406b540 msgr2=0x7f1fe4152200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:40.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:40.065+0000 7f1fe91bf700 1 --2- 172.21.15.43:0/181936966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fe406b540 0x7f1fe4152200 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1fd400c7f0 tx=0x7f1fd400cbb0 comp rx=0 tx=0).stop 2023-06-28T20:07:40.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:40.065+0000 7f1fe91bf700 1 -- 172.21.15.43:0/181936966 shutdown_connections 2023-06-28T20:07:40.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:40.065+0000 7f1fe91bf700 1 --2- 172.21.15.43:0/181936966 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1fc406ace0 0x7f1fc406d190 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:40.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:40.065+0000 7f1fe91bf700 1 --2- 172.21.15.43:0/181936966 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fe40d0630 0x7f1fe40b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:40.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:40.065+0000 7f1fe91bf700 1 --2- 172.21.15.43:0/181936966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fe406b540 0x7f1fe4152200 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:40.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:40.065+0000 7f1fe91bf700 1 --2- 172.21.15.43:0/181936966 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fe40cbd40 0x7f1fe415f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:40.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:40.065+0000 7f1fe91bf700 1 -- 172.21.15.43:0/181936966 >> 172.21.15.43:0/181936966 conn(0x7f1fe40664b0 msgr2=0x7f1fe40caf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:40.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:40.065+0000 7f1fe91bf700 1 -- 172.21.15.43:0/181936966 shutdown_connections 2023-06-28T20:07:40.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:40.065+0000 7f1fe91bf700 1 -- 172.21.15.43:0/181936966 wait complete. 2023-06-28T20:07:40.081 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-28T20:07:40.082 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-28T20:07:40.083 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2023-06-28T20:07:40.083 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-06-28T20:07:40.083 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-06-28T20:07:40.084 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-28T20:07:40.084 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-28T20:07:40.085 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-28T20:07:40.085 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-06-28T20:07:40.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:40.086 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2023-06-28T20:07:40.105 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-28T20:07:40.105 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-28T20:07:40.105 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2023-06-28T20:07:40.106 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:40.106 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2023-06-28T20:07:40.106 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:40.106 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-28T20:07:40.106 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2023-06-28T20:07:40.106 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-28T20:07:40.106 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2023-06-28T20:07:40.107 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-28T20:07:40.107 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2023-06-28T20:07:40.107 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-06-28T20:07:40.107 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 41899sec preferred_lft 41899sec 2023-06-28T20:07:40.107 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2023-06-28T20:07:40.107 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:40.107 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-28T20:07:40.108 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2023-06-28T20:07:40.108 INFO:teuthology.orchestra.run.smithi043.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-28T20:07:40.108 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 06:bd:a8:fa:4b:47 brd ff:ff:ff:ff:ff:ff 2023-06-28T20:07:40.108 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-06-28T20:07:40.108 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:40.108 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::86f:f5ff:fef2:e36f/64 scope link 2023-06-28T20:07:40.108 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:40.109 INFO:teuthology.orchestra.run.smithi043.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-28T20:07:40.109 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 06:bd:a8:fa:4b:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-06-28T20:07:40.109 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::4bd:a8ff:fefa:4b47/64 scope link tentative 2023-06-28T20:07:40.109 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:40.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:40.110 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-28T20:07:40.173 INFO:teuthology.orchestra.run.smithi043.stdout:1 2023-06-28T20:07:40.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:40.176 DEBUG:teuthology.orchestra.run.smithi043:> route 2023-06-28T20:07:40.233 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2023-06-28T20:07:40.234 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-28T20:07:40.234 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-06-28T20:07:40.234 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-06-28T20:07:40.234 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-28T20:07:40.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:40.235 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-06-28T20:07:40.235 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-06-28T20:07:40.235 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-06-28T20:07:40.235 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-06-28T20:07:40.236 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-06-28T20:07:40.236 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-06-28T20:07:40.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:40.340 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-06-28T20:07:40.357 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-06-28T20:07:40.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:40.358 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list-id 2023-06-28T20:07:40.414 INFO:teuthology.orchestra.run.smithi043.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-06-28T20:07:40.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:40.415 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-06-28T20:07:40.415 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-06-28T20:07:40.415 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-06-28T20:07:40.416 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-06-28T20:07:40.416 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-06-28T20:07:40.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:40.509 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-06-28T20:07:40.545 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-28T20:07:40.546 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-28T20:07:40.546 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2023-06-28T20:07:40.546 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:40.546 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2023-06-28T20:07:40.546 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:40.546 INFO:teuthology.orchestra.run.smithi043.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-28T20:07:40.547 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 6e:53:35:68:43:1c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-06-28T20:07:40.547 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-06-28T20:07:40.547 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:40.547 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::6c53:35ff:fe68:431c/64 scope link 2023-06-28T20:07:40.547 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:40.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:40.559 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-06-28T20:07:40.646 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-28T20:07:40.646 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-28T20:07:40.646 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2023-06-28T20:07:40.646 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:40.646 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2023-06-28T20:07:40.646 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:40.647 INFO:teuthology.orchestra.run.smithi043.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-28T20:07:40.647 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 6e:53:35:68:43:1c brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-06-28T20:07:40.647 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-06-28T20:07:40.647 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:40.647 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::6c53:35ff:fe68:431c/64 scope link 2023-06-28T20:07:40.648 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:40.661 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-06-28T20:07:40.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:40.662 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-06-28T20:07:40.662 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-06-28T20:07:40.663 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-06-28T20:07:40.663 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-06-28T20:07:40.663 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-06-28T20:07:40.663 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-06-28T20:07:40.664 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-06-28T20:07:40.664 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-06-28T20:07:40.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:40.880 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-06-28T20:07:40.880 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-06-28T20:07:40.880 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.1 up 2023-06-28T20:07:40.881 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set dev brx.1 master ceph-brx 2023-06-28T20:07:40.881 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-06-28T20:07:41.008 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-28T20:07:41.009 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-06-28T20:07:41.031 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-06-28T20:07:41.031 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-28T20:07:41.031 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-06-28T20:07:41.088 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-06-28T20:07:41.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:41.089 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-06-28T20:07:41.176 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-06-28T20:07:41.176 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-06-28T20:07:41.177 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2023-06-28T20:07:41.185 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-06-28T20:07:41.185 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-06-28T20:07:41.186 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2023-06-28T20:07:41.186 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.178:3300,172.21.15.178:3301" 2023-06-28T20:07:41.186 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2023-06-28T20:07:41.260 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-06-28T20:07:41.260 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-06-28T20:07:41.260 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2023-06-28T20:07:41.260 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.178:3300,172.21.15.178:3301" 2023-06-28T20:07:41.260 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2023-06-28T20:07:41.261 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.178:6789,172.21.15.178:6790" 2023-06-28T20:07:41.261 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2023-06-28T20:07:41.261 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=b8fd740d-00b9-4308-92a8-48f5fffe8078" will pass to kernel 2023-06-28T20:07:41.262 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-06-28T20:07:41.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:41.263 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-06-28T20:07:41.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:41.330 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-28T20:07:41.403 INFO:teuthology.orchestra.run.smithi043.stdout:client4469 2023-06-28T20:07:41.404 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:07:41.404 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:07:41.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:41.462 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-28T20:07:41.534 INFO:teuthology.orchestra.run.smithi043.stdout:client4469 2023-06-28T20:07:41.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:41.535 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-28T20:07:41.607 INFO:teuthology.orchestra.run.smithi043.stdout:client4469 2023-06-28T20:07:41.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:41.608 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/b8fd740d-00b9-4308-92a8-48f5fffe8078.client4469/status') 2023-06-28T20:07:41.688 INFO:teuthology.orchestra.run.smithi043.stdout:instance: client.4469 (3)172.21.15.43:0/2385900237 2023-06-28T20:07:41.688 INFO:teuthology.orchestra.run.smithi043.stdout:blocklisted: false 2023-06-28T20:07:41.689 INFO:teuthology.orchestra.run.smithi043.stderr:0+1 records in 2023-06-28T20:07:41.689 INFO:teuthology.orchestra.run.smithi043.stderr:0+1 records out 2023-06-28T20:07:41.689 INFO:teuthology.orchestra.run.smithi043.stderr:70 bytes copied, 3.2346e-05 s, 2.2 MB/s 2023-06-28T20:07:41.690 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-06-28T20:07:41.690 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-28T20:07:41.691 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-06-28T20:07:41.691 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-28T20:07:41.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.851+0000 7f12c9c88700 1 Processor -- start 2023-06-28T20:07:41.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.851+0000 7f12c9c88700 1 -- start start 2023-06-28T20:07:41.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.851+0000 7f12c9c88700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12c40cc3d0 0x7f12c415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:41.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.851+0000 7f12c9c88700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f12c40cb5c0 0x7f12c4071070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:41.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.851+0000 7f12c9c88700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c415e250 con 0x7f12c40cb1f0 2023-06-28T20:07:41.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.851+0000 7f12c9c88700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c415e040 con 0x7f12c40cb5c0 2023-06-28T20:07:41.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.851+0000 7f12c9c88700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c415f010 con 0x7f12c40cc3d0 2023-06-28T20:07:41.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.852+0000 7f12c3fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f12c40cb5c0 0x7f12c4071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:41.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.852+0000 7f12c37fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f12c40cb1f0 0x7f12c40cbcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42886/0 (socket says 172.21.15.43:42886) 2023-06-28T20:07:41.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.852+0000 7f12c37fe700 1 -- 172.21.15.43:0/3267587866 learned_addr learned my addr 172.21.15.43:0/3267587866 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:41.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.852+0000 7f12c2ffd700 1 --2- 172.21.15.43:0/3267587866 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12c40cc3d0 0x7f12c415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:41.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.852+0000 7f12c3fff700 1 -- 172.21.15.43:0/3267587866 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12c40cc3d0 msgr2=0x7f12c415b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:41.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.852+0000 7f12c3fff700 1 --2- 172.21.15.43:0/3267587866 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12c40cc3d0 0x7f12c415b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:41.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.852+0000 7f12c3fff700 1 -- 172.21.15.43:0/3267587866 >> v1:172.21.15.43:6789/0 conn(0x7f12c40cb1f0 legacy=0x7f12c40cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:41.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.852+0000 7f12c3fff700 1 -- 172.21.15.43:0/3267587866 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c415e530 con 0x7f12c40cb5c0 2023-06-28T20:07:41.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.852+0000 7f12c27fc700 1 -- 172.21.15.43:0/3267587866 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3461878288 0 0) 0x7f12c415e250 con 0x7f12c40cb1f0 2023-06-28T20:07:41.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.852+0000 7f12c3fff700 1 --2- 172.21.15.43:0/3267587866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c40cb5c0 0x7f12c4071070 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009840 tx=0x7f12bc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ed263a791fb7e7 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:41.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.852+0000 7f12c27fc700 1 -- 172.21.15.43:0/3267587866 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc011830 con 0x7f12c40cb5c0 2023-06-28T20:07:41.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.852+0000 7f12c27fc700 1 -- 172.21.15.43:0/3267587866 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12bc011990 con 0x7f12c40cb5c0 2023-06-28T20:07:41.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.852+0000 7f12c27fc700 1 -- 172.21.15.43:0/3267587866 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc011d20 con 0x7f12c40cb5c0 2023-06-28T20:07:41.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.853+0000 7f12c9c88700 1 -- 172.21.15.43:0/3267587866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c40cb5c0 msgr2=0x7f12c4071070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:41.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.853+0000 7f12c9c88700 1 --2- 172.21.15.43:0/3267587866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c40cb5c0 0x7f12c4071070 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009840 tx=0x7f12bc009b50 comp rx=0 tx=0).stop 2023-06-28T20:07:41.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.853+0000 7f12c9c88700 1 -- 172.21.15.43:0/3267587866 shutdown_connections 2023-06-28T20:07:41.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.853+0000 7f12c9c88700 1 --2- 172.21.15.43:0/3267587866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c40cb5c0 0x7f12c4071070 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:41.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.853+0000 7f12c9c88700 1 --2- 172.21.15.43:0/3267587866 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12c40cc3d0 0x7f12c415b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:41.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.853+0000 7f12c9c88700 1 -- 172.21.15.43:0/3267587866 >> 172.21.15.43:0/3267587866 conn(0x7f12c406c170 msgr2=0x7f12c406c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:41.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.853+0000 7f12c9c88700 1 -- 172.21.15.43:0/3267587866 shutdown_connections 2023-06-28T20:07:41.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.853+0000 7f12c9c88700 1 -- 172.21.15.43:0/3267587866 wait complete. 2023-06-28T20:07:41.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.854+0000 7f12c9c88700 1 Processor -- start 2023-06-28T20:07:41.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.854+0000 7f12c9c88700 1 -- start start 2023-06-28T20:07:41.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.855+0000 7f12c9c88700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12c40cb1f0 0x7f12c4151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:41.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.855+0000 7f12c9c88700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12c40cb5c0 0x7f12c40c3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:41.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.855+0000 7f12c9c88700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c40cc3d0 0x7f12c414f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:41.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.855+0000 7f12c9c88700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c40c4cd0 con 0x7f12c40cc3d0 2023-06-28T20:07:41.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.855+0000 7f12c9c88700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c40c3e40 con 0x7f12c40cb1f0 2023-06-28T20:07:41.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.855+0000 7f12c9c88700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c40c4050 con 0x7f12c40cb5c0 2023-06-28T20:07:41.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.855+0000 7f12c3fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c40cc3d0 0x7f12c414f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:41.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.855+0000 7f12c3fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c40cc3d0 0x7f12c414f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53706/0 (socket says 172.21.15.43:53706) 2023-06-28T20:07:41.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.855+0000 7f12c3fff700 1 -- 172.21.15.43:0/2961029540 learned_addr learned my addr 172.21.15.43:0/2961029540 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:41.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.855+0000 7f12c2ffd700 1 --2- 172.21.15.43:0/2961029540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12c40cb5c0 0x7f12c40c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:41.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.855+0000 7f12c37fe700 1 --2- 172.21.15.43:0/2961029540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12c40cb1f0 0x7f12c4151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:41.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.855+0000 7f12c3fff700 1 -- 172.21.15.43:0/2961029540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12c40cb5c0 msgr2=0x7f12c40c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:41.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.855+0000 7f12c3fff700 1 --2- 172.21.15.43:0/2961029540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12c40cb5c0 0x7f12c40c3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:41.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.855+0000 7f12c3fff700 1 -- 172.21.15.43:0/2961029540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12c40cb1f0 msgr2=0x7f12c4151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:41.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.855+0000 7f12c3fff700 1 --2- 172.21.15.43:0/2961029540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12c40cb1f0 0x7f12c4151610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:41.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.855+0000 7f12c3fff700 1 -- 172.21.15.43:0/2961029540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12bc0094a0 con 0x7f12c40cc3d0 2023-06-28T20:07:41.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.855+0000 7f12c37fe700 1 --2- 172.21.15.43:0/2961029540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12c40cb1f0 0x7f12c4151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:41.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.856+0000 7f12c3fff700 1 --2- 172.21.15.43:0/2961029540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c40cc3d0 0x7f12c414f960 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009fd0 tx=0x7f12bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:41.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.856+0000 7f12abfff700 1 -- 172.21.15.43:0/2961029540 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc0038a0 con 0x7f12c40cc3d0 2023-06-28T20:07:41.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.856+0000 7f12abfff700 1 -- 172.21.15.43:0/2961029540 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12bc01f070 con 0x7f12c40cc3d0 2023-06-28T20:07:41.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.856+0000 7f12abfff700 1 -- 172.21.15.43:0/2961029540 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc003c60 con 0x7f12c40cc3d0 2023-06-28T20:07:41.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.856+0000 7f12c9c88700 1 -- 172.21.15.43:0/2961029540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c40c4330 con 0x7f12c40cc3d0 2023-06-28T20:07:41.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.856+0000 7f12c9c88700 1 -- 172.21.15.43:0/2961029540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c40c48f0 con 0x7f12c40cc3d0 2023-06-28T20:07:41.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.857+0000 7f12c9c88700 1 -- 172.21.15.43:0/2961029540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c40c4e10 con 0x7f12c40cc3d0 2023-06-28T20:07:41.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.857+0000 7f12abfff700 1 -- 172.21.15.43:0/2961029540 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f12bc01a430 con 0x7f12c40cc3d0 2023-06-28T20:07:41.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.858+0000 7f12abfff700 1 --2- 172.21.15.43:0/2961029540 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f12b006ace0 0x7f12b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:41.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.858+0000 7f12abfff700 1 -- 172.21.15.43:0/2961029540 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f12bc021070 con 0x7f12c40cc3d0 2023-06-28T20:07:41.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.858+0000 7f12c37fe700 1 --2- 172.21.15.43:0/2961029540 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f12b006ace0 0x7f12b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:41.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.858+0000 7f12c37fe700 1 --2- 172.21.15.43:0/2961029540 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f12b006ace0 0x7f12b006d190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f12b40012e0 tx=0x7f12b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:41.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:41.862+0000 7f12abfff700 1 -- 172.21.15.43:0/2961029540 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f12bc055f10 con 0x7f12c40cc3d0 2023-06-28T20:07:42.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:42.109+0000 7f12c9c88700 1 -- 172.21.15.43:0/2961029540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f12c40728b0 con 0x7f12c40cc3d0 2023-06-28T20:07:42.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:42.109+0000 7f12abfff700 1 -- 172.21.15.43:0/2961029540 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f12bc015070 con 0x7f12c40cc3d0 2023-06-28T20:07:42.113 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-06-28T20:07:42.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:42.111+0000 7f12c9c88700 1 -- 172.21.15.43:0/2961029540 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f12b006ace0 msgr2=0x7f12b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:42.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:42.111+0000 7f12c9c88700 1 --2- 172.21.15.43:0/2961029540 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f12b006ace0 0x7f12b006d190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f12b40012e0 tx=0x7f12b4009040 comp rx=0 tx=0).stop 2023-06-28T20:07:42.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:42.111+0000 7f12c9c88700 1 -- 172.21.15.43:0/2961029540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c40cc3d0 msgr2=0x7f12c414f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:42.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:42.111+0000 7f12c9c88700 1 --2- 172.21.15.43:0/2961029540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c40cc3d0 0x7f12c414f960 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009fd0 tx=0x7f12bc004750 comp rx=0 tx=0).stop 2023-06-28T20:07:42.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:42.111+0000 7f12c9c88700 1 -- 172.21.15.43:0/2961029540 shutdown_connections 2023-06-28T20:07:42.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:42.111+0000 7f12c9c88700 1 --2- 172.21.15.43:0/2961029540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12c40cc3d0 0x7f12c414f960 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:42.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:42.111+0000 7f12c9c88700 1 --2- 172.21.15.43:0/2961029540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12c40cb1f0 0x7f12c4151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:42.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:42.111+0000 7f12c9c88700 1 --2- 172.21.15.43:0/2961029540 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f12b006ace0 0x7f12b006d190 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:42.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:42.111+0000 7f12c9c88700 1 --2- 172.21.15.43:0/2961029540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f12c40cb5c0 0x7f12c40c3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:42.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:42.112+0000 7f12c9c88700 1 -- 172.21.15.43:0/2961029540 >> 172.21.15.43:0/2961029540 conn(0x7f12c406c170 msgr2=0x7f12c40cbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:42.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:42.112+0000 7f12c9c88700 1 -- 172.21.15.43:0/2961029540 shutdown_connections 2023-06-28T20:07:42.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:42.112+0000 7f12c9c88700 1 -- 172.21.15.43:0/2961029540 wait complete. 2023-06-28T20:07:42.127 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-28T20:07:42.128 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-28T20:07:42.128 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi178.front.sepia.ceph.com 2023-06-28T20:07:42.129 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-06-28T20:07:42.129 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-06-28T20:07:42.129 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-28T20:07:42.129 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-28T20:07:42.129 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-28T20:07:42.130 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-06-28T20:07:42.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:42.130 DEBUG:teuthology.orchestra.run.smithi178:> ip addr 2023-06-28T20:07:42.152 INFO:teuthology.orchestra.run.smithi178.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-28T20:07:42.152 INFO:teuthology.orchestra.run.smithi178.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-28T20:07:42.152 INFO:teuthology.orchestra.run.smithi178.stdout: inet 127.0.0.1/8 scope host lo 2023-06-28T20:07:42.152 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:42.152 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 ::1/128 scope host 2023-06-28T20:07:42.153 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:42.153 INFO:teuthology.orchestra.run.smithi178.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-28T20:07:42.153 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:d9:3b:20 brd ff:ff:ff:ff:ff:ff 2023-06-28T20:07:42.153 INFO:teuthology.orchestra.run.smithi178.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-28T20:07:42.153 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:d9:3b:21 brd ff:ff:ff:ff:ff:ff 2023-06-28T20:07:42.153 INFO:teuthology.orchestra.run.smithi178.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-28T20:07:42.153 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:88:7e:f8 brd ff:ff:ff:ff:ff:ff 2023-06-28T20:07:42.154 INFO:teuthology.orchestra.run.smithi178.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-28T20:07:42.154 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:88:7e:f9 brd ff:ff:ff:ff:ff:ff 2023-06-28T20:07:42.154 INFO:teuthology.orchestra.run.smithi178.stdout: inet 172.21.15.178/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-06-28T20:07:42.154 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft 41955sec preferred_lft 41955sec 2023-06-28T20:07:42.154 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::6f8b:bfa0:af97:1715/64 scope link noprefixroute 2023-06-28T20:07:42.154 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:42.155 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-06-28T20:07:42.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:42.155 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec bash -c ' 2023-06-28T20:07:42.155 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2023-06-28T20:07:42.156 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link add name ceph-brx type bridge 2023-06-28T20:07:42.156 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip addr flush dev ceph-brx 2023-06-28T20:07:42.156 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set ceph-brx up 2023-06-28T20:07:42.156 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-06-28T20:07:42.157 DEBUG:teuthology.orchestra.run.smithi178:> ') 2023-06-28T20:07:42.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:42.335 DEBUG:teuthology.orchestra.run.smithi178:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-28T20:07:42.369 INFO:teuthology.orchestra.run.smithi178.stdout:1 2023-06-28T20:07:42.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:42.378 DEBUG:teuthology.orchestra.run.smithi178:> route 2023-06-28T20:07:42.450 INFO:teuthology.orchestra.run.smithi178.stdout:Kernel IP routing table 2023-06-28T20:07:42.451 INFO:teuthology.orchestra.run.smithi178.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-28T20:07:42.451 INFO:teuthology.orchestra.run.smithi178.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-06-28T20:07:42.451 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-06-28T20:07:42.451 INFO:teuthology.orchestra.run.smithi178.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-28T20:07:42.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:42.452 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec bash -c ' 2023-06-28T20:07:42.452 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2023-06-28T20:07:42.453 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-06-28T20:07:42.453 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-06-28T20:07:42.453 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-06-28T20:07:42.454 DEBUG:teuthology.orchestra.run.smithi178:> ') 2023-06-28T20:07:42.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:42.711 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2023-06-28T20:07:42.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:42.730 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list-id 2023-06-28T20:07:42.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:42.788 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec bash -c ' 2023-06-28T20:07:42.788 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2023-06-28T20:07:42.788 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-06-28T20:07:42.789 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-06-28T20:07:42.789 DEBUG:teuthology.orchestra.run.smithi178:> ') 2023-06-28T20:07:42.889 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-06-28T20:07:42.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:42.890 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec bash -c ' 2023-06-28T20:07:42.891 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2023-06-28T20:07:42.891 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-06-28T20:07:42.891 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-06-28T20:07:42.892 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-06-28T20:07:42.892 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-06-28T20:07:42.892 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-06-28T20:07:42.893 DEBUG:teuthology.orchestra.run.smithi178:> ') 2023-06-28T20:07:43.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:43.085 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec bash -c ' 2023-06-28T20:07:43.085 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2023-06-28T20:07:43.085 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set brx.0 up 2023-06-28T20:07:43.086 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set dev brx.0 master ceph-brx 2023-06-28T20:07:43.086 DEBUG:teuthology.orchestra.run.smithi178:> ') 2023-06-28T20:07:43.191 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-28T20:07:43.191 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-06-28T20:07:43.215 INFO:teuthology.orchestra.run.smithi178.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-06-28T20:07:43.216 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-28T20:07:43.216 DEBUG:teuthology.orchestra.run.smithi178:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-06-28T20:07:43.272 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-06-28T20:07:43.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:43.272 DEBUG:teuthology.orchestra.run.smithi178:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-06-28T20:07:43.371 INFO:teuthology.orchestra.run.smithi178.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-06-28T20:07:43.371 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-06-28T20:07:43.371 INFO:teuthology.orchestra.run.smithi178.stdout:invalid new device string format 2023-06-28T20:07:43.467 INFO:teuthology.orchestra.run.smithi178.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-06-28T20:07:43.467 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-06-28T20:07:43.467 INFO:teuthology.orchestra.run.smithi178.stdout:invalid new device string format 2023-06-28T20:07:43.468 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.178:3300,172.21.15.178:3301" 2023-06-28T20:07:43.468 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: switching to using v1 address with old syntax 2023-06-28T20:07:43.496 INFO:teuthology.orchestra.run.smithi178.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-06-28T20:07:43.496 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-06-28T20:07:43.497 INFO:teuthology.orchestra.run.smithi178.stdout:invalid new device string format 2023-06-28T20:07:43.497 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.178:3300,172.21.15.178:3301" 2023-06-28T20:07:43.497 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: switching to using v1 address with old syntax 2023-06-28T20:07:43.497 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.178:6789,172.21.15.178:6790" 2023-06-28T20:07:43.497 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2023-06-28T20:07:43.497 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=b8fd740d-00b9-4308-92a8-48f5fffe8078" will pass to kernel 2023-06-28T20:07:43.513 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-06-28T20:07:43.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:43.514 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-06-28T20:07:43.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:43.540 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-28T20:07:43.611 INFO:teuthology.orchestra.run.smithi178.stdout:client4478 2023-06-28T20:07:43.612 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:07:43.612 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:07:43.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:43.671 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-28T20:07:43.742 INFO:teuthology.orchestra.run.smithi178.stdout:client4478 2023-06-28T20:07:43.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:43.743 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-28T20:07:43.814 INFO:teuthology.orchestra.run.smithi178.stdout:client4478 2023-06-28T20:07:43.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:43.815 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/b8fd740d-00b9-4308-92a8-48f5fffe8078.client4478/status') 2023-06-28T20:07:43.895 INFO:teuthology.orchestra.run.smithi178.stdout:instance: client.4478 (3)192.168.0.1:0/687981272 2023-06-28T20:07:43.895 INFO:teuthology.orchestra.run.smithi178.stdout:blocklisted: false 2023-06-28T20:07:43.896 INFO:teuthology.orchestra.run.smithi178.stderr:0+1 records in 2023-06-28T20:07:43.896 INFO:teuthology.orchestra.run.smithi178.stderr:0+1 records out 2023-06-28T20:07:43.896 INFO:teuthology.orchestra.run.smithi178.stderr:68 bytes copied, 2.1837e-05 s, 3.1 MB/s 2023-06-28T20:07:43.897 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-06-28T20:07:43.897 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-28T20:07:43.897 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-06-28T20:07:43.897 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-28T20:07:44.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.034+0000 7f0f45fbc700 1 Processor -- start 2023-06-28T20:07:44.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.034+0000 7f0f45fbc700 1 -- start start 2023-06-28T20:07:44.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.035+0000 7f0f45fbc700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f400d10f0 0x7f0f400ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:44.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.035+0000 7f0f45fbc700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4015ab50 con 0x7f0f400cedf0 2023-06-28T20:07:44.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.035+0000 7f0f45fbc700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f400cd1b0 con 0x7f0f400cbab0 2023-06-28T20:07:44.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.035+0000 7f0f45fbc700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f400cbf30 con 0x7f0f400d10f0 2023-06-28T20:07:44.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.035+0000 7f0f3effd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0f400cedf0 0x7f0f400cb3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42896/0 (socket says 172.21.15.43:42896) 2023-06-28T20:07:44.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.035+0000 7f0f3effd700 1 -- 172.21.15.43:0/4007961908 learned_addr learned my addr 172.21.15.43:0/4007961908 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:44.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.035+0000 7f0f3f7fe700 1 --2- 172.21.15.43:0/4007961908 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f400d10f0 0x7f0f400ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:44.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.035+0000 7f0f3e7fc700 1 -- 172.21.15.43:0/4007961908 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2039954956 0 0) 0x7f0f4015ab50 con 0x7f0f400cedf0 2023-06-28T20:07:44.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.035+0000 7f0f3e7fc700 1 -- 172.21.15.43:0/4007961908 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f280035b0 con 0x7f0f400cedf0 2023-06-28T20:07:44.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.035+0000 7f0f3e7fc700 1 -- 172.21.15.43:0/4007961908 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 807193564 0 0) 0x7f0f400cd1b0 con 0x7f0f400cbab0 2023-06-28T20:07:44.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.035+0000 7f0f3e7fc700 1 -- 172.21.15.43:0/4007961908 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f4015ab50 con 0x7f0f400cbab0 2023-06-28T20:07:44.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.035+0000 7f0f3f7fe700 1 -- 172.21.15.43:0/4007961908 >> v1:172.21.15.178:6789/0 conn(0x7f0f400cbab0 legacy=0x7f0f400b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:44.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.035+0000 7f0f3f7fe700 1 -- 172.21.15.43:0/4007961908 >> v1:172.21.15.43:6789/0 conn(0x7f0f400cedf0 legacy=0x7f0f400cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:44.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.035+0000 7f0f3f7fe700 1 -- 172.21.15.43:0/4007961908 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f400cc110 con 0x7f0f400d10f0 2023-06-28T20:07:44.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.035+0000 7f0f3f7fe700 1 --2- 172.21.15.43:0/4007961908 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f400d10f0 0x7f0f400ce8b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009850 tx=0x7f0f30009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f956c14df8029c71 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:44.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.036+0000 7f0f3e7fc700 1 -- 172.21.15.43:0/4007961908 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f30005e30 con 0x7f0f400d10f0 2023-06-28T20:07:44.045 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.036+0000 7f0f3e7fc700 1 -- 172.21.15.43:0/4007961908 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f30006030 con 0x7f0f400d10f0 2023-06-28T20:07:44.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.036+0000 7f0f3e7fc700 1 -- 172.21.15.43:0/4007961908 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f300063e0 con 0x7f0f400d10f0 2023-06-28T20:07:44.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.036+0000 7f0f45fbc700 1 -- 172.21.15.43:0/4007961908 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f400d10f0 msgr2=0x7f0f400ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:44.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.036+0000 7f0f45fbc700 1 --2- 172.21.15.43:0/4007961908 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f400d10f0 0x7f0f400ce8b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0f30009850 tx=0x7f0f30009b60 comp rx=0 tx=0).stop 2023-06-28T20:07:44.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.036+0000 7f0f45fbc700 1 -- 172.21.15.43:0/4007961908 shutdown_connections 2023-06-28T20:07:44.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.036+0000 7f0f45fbc700 1 --2- 172.21.15.43:0/4007961908 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f400d10f0 0x7f0f400ce8b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:44.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.036+0000 7f0f45fbc700 1 -- 172.21.15.43:0/4007961908 >> 172.21.15.43:0/4007961908 conn(0x7f0f400664b0 msgr2=0x7f0f4009a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:44.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.036+0000 7f0f45fbc700 1 -- 172.21.15.43:0/4007961908 shutdown_connections 2023-06-28T20:07:44.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.036+0000 7f0f45fbc700 1 -- 172.21.15.43:0/4007961908 wait complete. 2023-06-28T20:07:44.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.037+0000 7f0f45fbc700 1 Processor -- start 2023-06-28T20:07:44.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.038+0000 7f0f45fbc700 1 -- start start 2023-06-28T20:07:44.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.038+0000 7f0f45fbc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f400cbab0 0x7f0f401f1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:44.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.038+0000 7f0f45fbc700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f400cedf0 0x7f0f400b60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:44.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.038+0000 7f0f45fbc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f400d10f0 0x7f0f400b6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:44.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.038+0000 7f0f45fbc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4015db60 con 0x7f0f400cbab0 2023-06-28T20:07:44.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.038+0000 7f0f45fbc700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f401489c0 con 0x7f0f400d10f0 2023-06-28T20:07:44.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.038+0000 7f0f45fbc700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4015d6d0 con 0x7f0f400cedf0 2023-06-28T20:07:44.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.038+0000 7f0f3f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f400cbab0 0x7f0f401f1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:44.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.039+0000 7f0f3f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f400cbab0 0x7f0f401f1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53718/0 (socket says 172.21.15.43:53718) 2023-06-28T20:07:44.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.039+0000 7f0f3f7fe700 1 -- 172.21.15.43:0/3902705436 learned_addr learned my addr 172.21.15.43:0/3902705436 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:44.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.039+0000 7f0f3effd700 1 --2- 172.21.15.43:0/3902705436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f400cedf0 0x7f0f400b60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:44.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.039+0000 7f0f3ffff700 1 --2- 172.21.15.43:0/3902705436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f400d10f0 0x7f0f400b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:44.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.039+0000 7f0f3f7fe700 1 -- 172.21.15.43:0/3902705436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f400cedf0 msgr2=0x7f0f400b60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:44.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.039+0000 7f0f3f7fe700 1 --2- 172.21.15.43:0/3902705436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f400cedf0 0x7f0f400b60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:44.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.039+0000 7f0f3f7fe700 1 -- 172.21.15.43:0/3902705436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f400d10f0 msgr2=0x7f0f400b6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:44.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.039+0000 7f0f3f7fe700 1 --2- 172.21.15.43:0/3902705436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f400d10f0 0x7f0f400b6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:44.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.039+0000 7f0f3f7fe700 1 -- 172.21.15.43:0/3902705436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f30009430 con 0x7f0f400cbab0 2023-06-28T20:07:44.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.039+0000 7f0f3f7fe700 1 --2- 172.21.15.43:0/3902705436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f400cbab0 0x7f0f401f1720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0f30000c00 tx=0x7f0f30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:44.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.039+0000 7f0f27fff700 1 -- 172.21.15.43:0/3902705436 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f30006790 con 0x7f0f400cbab0 2023-06-28T20:07:44.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.039+0000 7f0f27fff700 1 -- 172.21.15.43:0/3902705436 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f30020070 con 0x7f0f400cbab0 2023-06-28T20:07:44.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.039+0000 7f0f45fbc700 1 -- 172.21.15.43:0/3902705436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f4015cda0 con 0x7f0f400cbab0 2023-06-28T20:07:44.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.039+0000 7f0f27fff700 1 -- 172.21.15.43:0/3902705436 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f30025670 con 0x7f0f400cbab0 2023-06-28T20:07:44.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.039+0000 7f0f45fbc700 1 -- 172.21.15.43:0/3902705436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f4015d360 con 0x7f0f400cbab0 2023-06-28T20:07:44.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.040+0000 7f0f45fbc700 1 -- 172.21.15.43:0/3902705436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f400d0080 con 0x7f0f400cbab0 2023-06-28T20:07:44.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.041+0000 7f0f27fff700 1 -- 172.21.15.43:0/3902705436 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0f30022030 con 0x7f0f400cbab0 2023-06-28T20:07:44.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.041+0000 7f0f27fff700 1 --2- 172.21.15.43:0/3902705436 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0f2c06ace0 0x7f0f2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:44.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.041+0000 7f0f27fff700 1 -- 172.21.15.43:0/3902705436 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0f3008a7e0 con 0x7f0f400cbab0 2023-06-28T20:07:44.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.041+0000 7f0f3effd700 1 --2- 172.21.15.43:0/3902705436 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0f2c06ace0 0x7f0f2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:44.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.042+0000 7f0f3effd700 1 --2- 172.21.15.43:0/3902705436 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0f2c06ace0 0x7f0f2c06d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0f40158da0 tx=0x7f0f34007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:44.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.046+0000 7f0f27fff700 1 -- 172.21.15.43:0/3902705436 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0f30055920 con 0x7f0f400cbab0 2023-06-28T20:07:44.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.295+0000 7f0f45fbc700 1 -- 172.21.15.43:0/3902705436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0f400728b0 con 0x7f0f400cbab0 2023-06-28T20:07:44.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.296+0000 7f0f27fff700 1 -- 172.21.15.43:0/3902705436 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0f30058f40 con 0x7f0f400cbab0 2023-06-28T20:07:44.299 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-06-28T20:07:44.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.298+0000 7f0f45fbc700 1 -- 172.21.15.43:0/3902705436 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0f2c06ace0 msgr2=0x7f0f2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:44.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.298+0000 7f0f45fbc700 1 --2- 172.21.15.43:0/3902705436 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0f2c06ace0 0x7f0f2c06d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0f40158da0 tx=0x7f0f34007de0 comp rx=0 tx=0).stop 2023-06-28T20:07:44.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.298+0000 7f0f45fbc700 1 -- 172.21.15.43:0/3902705436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f400cbab0 msgr2=0x7f0f401f1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:44.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.298+0000 7f0f45fbc700 1 --2- 172.21.15.43:0/3902705436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f400cbab0 0x7f0f401f1720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0f30000c00 tx=0x7f0f30004750 comp rx=0 tx=0).stop 2023-06-28T20:07:44.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.298+0000 7f0f45fbc700 1 -- 172.21.15.43:0/3902705436 shutdown_connections 2023-06-28T20:07:44.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.298+0000 7f0f45fbc700 1 --2- 172.21.15.43:0/3902705436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f400d10f0 0x7f0f400b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:44.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.298+0000 7f0f45fbc700 1 --2- 172.21.15.43:0/3902705436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f400cbab0 0x7f0f401f1720 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:44.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.298+0000 7f0f45fbc700 1 --2- 172.21.15.43:0/3902705436 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0f2c06ace0 0x7f0f2c06d190 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:44.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.298+0000 7f0f45fbc700 1 --2- 172.21.15.43:0/3902705436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f400cedf0 0x7f0f400b60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:44.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.298+0000 7f0f45fbc700 1 -- 172.21.15.43:0/3902705436 >> 172.21.15.43:0/3902705436 conn(0x7f0f400664b0 msgr2=0x7f0f4006dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:44.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.298+0000 7f0f45fbc700 1 -- 172.21.15.43:0/3902705436 shutdown_connections 2023-06-28T20:07:44.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:44.298+0000 7f0f45fbc700 1 -- 172.21.15.43:0/3902705436 wait complete. 2023-06-28T20:07:44.313 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-28T20:07:44.314 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-28T20:07:44.314 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi178.front.sepia.ceph.com 2023-06-28T20:07:44.314 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-06-28T20:07:44.315 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-06-28T20:07:44.315 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-28T20:07:44.315 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-28T20:07:44.316 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-28T20:07:44.316 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-06-28T20:07:44.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:44.316 DEBUG:teuthology.orchestra.run.smithi178:> ip addr 2023-06-28T20:07:44.334 INFO:teuthology.orchestra.run.smithi178.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-28T20:07:44.335 INFO:teuthology.orchestra.run.smithi178.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-28T20:07:44.335 INFO:teuthology.orchestra.run.smithi178.stdout: inet 127.0.0.1/8 scope host lo 2023-06-28T20:07:44.335 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:44.335 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 ::1/128 scope host 2023-06-28T20:07:44.336 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:44.336 INFO:teuthology.orchestra.run.smithi178.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-28T20:07:44.336 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:d9:3b:20 brd ff:ff:ff:ff:ff:ff 2023-06-28T20:07:44.336 INFO:teuthology.orchestra.run.smithi178.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-28T20:07:44.336 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:d9:3b:21 brd ff:ff:ff:ff:ff:ff 2023-06-28T20:07:44.336 INFO:teuthology.orchestra.run.smithi178.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-28T20:07:44.336 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:88:7e:f8 brd ff:ff:ff:ff:ff:ff 2023-06-28T20:07:44.337 INFO:teuthology.orchestra.run.smithi178.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-28T20:07:44.337 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:88:7e:f9 brd ff:ff:ff:ff:ff:ff 2023-06-28T20:07:44.337 INFO:teuthology.orchestra.run.smithi178.stdout: inet 172.21.15.178/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-06-28T20:07:44.337 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft 41953sec preferred_lft 41953sec 2023-06-28T20:07:44.337 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::6f8b:bfa0:af97:1715/64 scope link noprefixroute 2023-06-28T20:07:44.337 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:44.337 INFO:teuthology.orchestra.run.smithi178.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-28T20:07:44.338 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether ee:06:1c:61:d8:26 brd ff:ff:ff:ff:ff:ff 2023-06-28T20:07:44.338 INFO:teuthology.orchestra.run.smithi178.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-06-28T20:07:44.338 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:44.338 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::4031:96ff:fe27:8efd/64 scope link 2023-06-28T20:07:44.338 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:44.338 INFO:teuthology.orchestra.run.smithi178.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-28T20:07:44.338 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether ee:06:1c:61:d8:26 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-06-28T20:07:44.339 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::ec06:1cff:fe61:d826/64 scope link 2023-06-28T20:07:44.339 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:44.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:44.339 DEBUG:teuthology.orchestra.run.smithi178:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-28T20:07:44.400 INFO:teuthology.orchestra.run.smithi178.stdout:1 2023-06-28T20:07:44.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:44.401 DEBUG:teuthology.orchestra.run.smithi178:> route 2023-06-28T20:07:44.460 INFO:teuthology.orchestra.run.smithi178.stdout:Kernel IP routing table 2023-06-28T20:07:44.461 INFO:teuthology.orchestra.run.smithi178.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-28T20:07:44.461 INFO:teuthology.orchestra.run.smithi178.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-06-28T20:07:44.461 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-06-28T20:07:44.461 INFO:teuthology.orchestra.run.smithi178.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-28T20:07:44.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:44.462 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec bash -c ' 2023-06-28T20:07:44.462 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2023-06-28T20:07:44.463 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-06-28T20:07:44.463 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-06-28T20:07:44.463 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-06-28T20:07:44.463 DEBUG:teuthology.orchestra.run.smithi178:> ') 2023-06-28T20:07:44.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:44.565 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2023-06-28T20:07:44.581 INFO:teuthology.orchestra.run.smithi178.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-06-28T20:07:44.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:44.583 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list-id 2023-06-28T20:07:44.638 INFO:teuthology.orchestra.run.smithi178.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-06-28T20:07:44.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:44.639 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec bash -c ' 2023-06-28T20:07:44.639 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2023-06-28T20:07:44.640 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-06-28T20:07:44.640 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-06-28T20:07:44.640 DEBUG:teuthology.orchestra.run.smithi178:> ') 2023-06-28T20:07:44.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:44.731 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-06-28T20:07:44.766 INFO:teuthology.orchestra.run.smithi178.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-28T20:07:44.767 INFO:teuthology.orchestra.run.smithi178.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-28T20:07:44.767 INFO:teuthology.orchestra.run.smithi178.stdout: inet 127.0.0.1/8 scope host lo 2023-06-28T20:07:44.767 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:44.767 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 ::1/128 scope host 2023-06-28T20:07:44.768 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:44.768 INFO:teuthology.orchestra.run.smithi178.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-28T20:07:44.768 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 5a:42:18:dd:c8:67 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-06-28T20:07:44.768 INFO:teuthology.orchestra.run.smithi178.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-06-28T20:07:44.768 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:44.768 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::5842:18ff:fedd:c867/64 scope link 2023-06-28T20:07:44.768 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:44.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:44.779 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-06-28T20:07:44.858 INFO:teuthology.orchestra.run.smithi178.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-28T20:07:44.859 INFO:teuthology.orchestra.run.smithi178.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-28T20:07:44.859 INFO:teuthology.orchestra.run.smithi178.stdout: inet 127.0.0.1/8 scope host lo 2023-06-28T20:07:44.859 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:44.859 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 ::1/128 scope host 2023-06-28T20:07:44.859 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:44.859 INFO:teuthology.orchestra.run.smithi178.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-28T20:07:44.859 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 5a:42:18:dd:c8:67 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-06-28T20:07:44.860 INFO:teuthology.orchestra.run.smithi178.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-06-28T20:07:44.860 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:44.860 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::5842:18ff:fedd:c867/64 scope link 2023-06-28T20:07:44.860 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:07:44.878 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-06-28T20:07:44.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:44.879 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec bash -c ' 2023-06-28T20:07:44.880 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2023-06-28T20:07:44.880 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-06-28T20:07:44.880 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-06-28T20:07:44.880 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-06-28T20:07:44.880 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-06-28T20:07:44.881 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-06-28T20:07:44.881 DEBUG:teuthology.orchestra.run.smithi178:> ') 2023-06-28T20:07:45.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:45.124 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec bash -c ' 2023-06-28T20:07:45.124 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2023-06-28T20:07:45.125 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set brx.1 up 2023-06-28T20:07:45.125 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set dev brx.1 master ceph-brx 2023-06-28T20:07:45.125 DEBUG:teuthology.orchestra.run.smithi178:> ') 2023-06-28T20:07:45.239 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-28T20:07:45.240 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-06-28T20:07:45.260 INFO:teuthology.orchestra.run.smithi178.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-06-28T20:07:45.261 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-28T20:07:45.261 DEBUG:teuthology.orchestra.run.smithi178:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-06-28T20:07:45.316 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-06-28T20:07:45.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:45.316 DEBUG:teuthology.orchestra.run.smithi178:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-06-28T20:07:45.401 INFO:teuthology.orchestra.run.smithi178.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-06-28T20:07:45.402 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-06-28T20:07:45.402 INFO:teuthology.orchestra.run.smithi178.stdout:invalid new device string format 2023-06-28T20:07:45.410 INFO:teuthology.orchestra.run.smithi178.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-06-28T20:07:45.411 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-06-28T20:07:45.411 INFO:teuthology.orchestra.run.smithi178.stdout:invalid new device string format 2023-06-28T20:07:45.411 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.178:3300,172.21.15.178:3301" 2023-06-28T20:07:45.411 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: switching to using v1 address with old syntax 2023-06-28T20:07:45.442 INFO:teuthology.orchestra.run.smithi178.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-06-28T20:07:45.442 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-06-28T20:07:45.442 INFO:teuthology.orchestra.run.smithi178.stdout:invalid new device string format 2023-06-28T20:07:45.443 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.178:3300,172.21.15.178:3301" 2023-06-28T20:07:45.443 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: switching to using v1 address with old syntax 2023-06-28T20:07:45.443 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.178:6789,172.21.15.178:6790" 2023-06-28T20:07:45.443 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2023-06-28T20:07:45.443 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=b8fd740d-00b9-4308-92a8-48f5fffe8078" will pass to kernel 2023-06-28T20:07:45.461 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-06-28T20:07:45.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:45.463 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-06-28T20:07:45.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:45.491 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-28T20:07:45.562 INFO:teuthology.orchestra.run.smithi178.stdout:client4487 2023-06-28T20:07:45.563 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:07:45.563 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:07:45.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:45.621 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-28T20:07:45.693 INFO:teuthology.orchestra.run.smithi178.stdout:client4487 2023-06-28T20:07:45.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:45.695 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-28T20:07:45.766 INFO:teuthology.orchestra.run.smithi178.stdout:client4487 2023-06-28T20:07:45.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:45.767 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/b8fd740d-00b9-4308-92a8-48f5fffe8078.client4487/status') 2023-06-28T20:07:45.845 INFO:teuthology.orchestra.run.smithi178.stdout:instance: client.4487 (3)192.168.0.2:0/4212045604 2023-06-28T20:07:45.846 INFO:teuthology.orchestra.run.smithi178.stdout:blocklisted: false 2023-06-28T20:07:45.846 INFO:teuthology.orchestra.run.smithi178.stderr:0+1 records in 2023-06-28T20:07:45.847 INFO:teuthology.orchestra.run.smithi178.stderr:0+1 records out 2023-06-28T20:07:45.847 INFO:teuthology.orchestra.run.smithi178.stderr:69 bytes copied, 2.2885e-05 s, 3.0 MB/s 2023-06-28T20:07:45.847 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-06-28T20:07:45.865 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:07:46.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.001+0000 7f235cafd700 1 Processor -- start 2023-06-28T20:07:46.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.001+0000 7f235cafd700 1 -- start start 2023-06-28T20:07:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.002+0000 7f235cafd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f23580d0630 0x7f23580d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.002+0000 7f235cafd700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23580ccf10 con 0x7f235806b710 2023-06-28T20:07:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.002+0000 7f235cafd700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f235815cd80 con 0x7f23580cbd40 2023-06-28T20:07:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.002+0000 7f235cafd700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23580d0a00 con 0x7f23580d0630 2023-06-28T20:07:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.002+0000 7f235659c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f23580d0630 0x7f23580d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.002+0000 7f2356d9d700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f235806b710 0x7f23580ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42916/0 (socket says 172.21.15.43:42916) 2023-06-28T20:07:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.002+0000 7f2356d9d700 1 -- 172.21.15.43:0/1928707603 learned_addr learned my addr 172.21.15.43:0/1928707603 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:46.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.002+0000 7f235659c700 1 -- 172.21.15.43:0/1928707603 >> v1:172.21.15.178:6789/0 conn(0x7f23580cbd40 legacy=0x7f235806b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:46.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.002+0000 7f235659c700 1 -- 172.21.15.43:0/1928707603 >> v1:172.21.15.43:6789/0 conn(0x7f235806b710 legacy=0x7f23580ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:46.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.002+0000 7f235659c700 1 -- 172.21.15.43:0/1928707603 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23580cc300 con 0x7f23580d0630 2023-06-28T20:07:46.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.002+0000 7f235559a700 1 -- 172.21.15.43:0/1928707603 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000106710 0 0) 0x7f23580ccf10 con 0x7f235806b710 2023-06-28T20:07:46.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.003+0000 7f235659c700 1 --2- 172.21.15.43:0/1928707603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23580d0630 0x7f23580d1070 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2348009850 tx=0x7f2348009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd97757ccee9bccc server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:46.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.003+0000 7f235559a700 1 -- 172.21.15.43:0/1928707603 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2348005e30 con 0x7f23580d0630 2023-06-28T20:07:46.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.003+0000 7f235559a700 1 -- 172.21.15.43:0/1928707603 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2348006030 con 0x7f23580d0630 2023-06-28T20:07:46.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.003+0000 7f235559a700 1 -- 172.21.15.43:0/1928707603 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23480063c0 con 0x7f23580d0630 2023-06-28T20:07:46.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.003+0000 7f235cafd700 1 -- 172.21.15.43:0/1928707603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23580d0630 msgr2=0x7f23580d1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:46.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.003+0000 7f235cafd700 1 --2- 172.21.15.43:0/1928707603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23580d0630 0x7f23580d1070 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2348009850 tx=0x7f2348009b60 comp rx=0 tx=0).stop 2023-06-28T20:07:46.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.003+0000 7f235cafd700 1 -- 172.21.15.43:0/1928707603 shutdown_connections 2023-06-28T20:07:46.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.003+0000 7f235cafd700 1 --2- 172.21.15.43:0/1928707603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23580d0630 0x7f23580d1070 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.003+0000 7f235cafd700 1 -- 172.21.15.43:0/1928707603 >> 172.21.15.43:0/1928707603 conn(0x7f23580664b0 msgr2=0x7f235809a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:46.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.003+0000 7f235cafd700 1 -- 172.21.15.43:0/1928707603 shutdown_connections 2023-06-28T20:07:46.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.003+0000 7f235cafd700 1 -- 172.21.15.43:0/1928707603 wait complete. 2023-06-28T20:07:46.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.005+0000 7f235cafd700 1 Processor -- start 2023-06-28T20:07:46.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.005+0000 7f235cafd700 1 -- start start 2023-06-28T20:07:46.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.005+0000 7f235cafd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235806b710 0x7f23580c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:46.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.005+0000 7f235cafd700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23580cbd40 0x7f23580c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:46.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.006+0000 7f235cafd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23580d0630 0x7f23580c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:46.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.006+0000 7f235cafd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23580b0110 con 0x7f235806b710 2023-06-28T20:07:46.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.006+0000 7f235cafd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23580b03f0 con 0x7f23580d0630 2023-06-28T20:07:46.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.006+0000 7f235cafd700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23580b06d0 con 0x7f23580cbd40 2023-06-28T20:07:46.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.006+0000 7f235659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235806b710 0x7f23580c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:46.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.006+0000 7f235659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235806b710 0x7f23580c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53732/0 (socket says 172.21.15.43:53732) 2023-06-28T20:07:46.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.006+0000 7f235659c700 1 -- 172.21.15.43:0/1496336891 learned_addr learned my addr 172.21.15.43:0/1496336891 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:46.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.006+0000 7f2356d9d700 1 --2- 172.21.15.43:0/1496336891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23580d0630 0x7f23580c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:46.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.006+0000 7f2355d9b700 1 --2- 172.21.15.43:0/1496336891 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23580cbd40 0x7f23580c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:46.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.006+0000 7f235659c700 1 -- 172.21.15.43:0/1496336891 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23580cbd40 msgr2=0x7f23580c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:46.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.006+0000 7f235659c700 1 --2- 172.21.15.43:0/1496336891 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23580cbd40 0x7f23580c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.006+0000 7f235659c700 1 -- 172.21.15.43:0/1496336891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23580d0630 msgr2=0x7f23580c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:46.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.006+0000 7f235659c700 1 --2- 172.21.15.43:0/1496336891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23580d0630 0x7f23580c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.006+0000 7f235659c700 1 -- 172.21.15.43:0/1496336891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2348009430 con 0x7f235806b710 2023-06-28T20:07:46.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.006+0000 7f235659c700 1 --2- 172.21.15.43:0/1496336891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235806b710 0x7f23580c7db0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2348013040 tx=0x7f2348004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:46.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.006+0000 7f2346ffd700 1 -- 172.21.15.43:0/1496336891 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2348025430 con 0x7f235806b710 2023-06-28T20:07:46.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.006+0000 7f2346ffd700 1 -- 172.21.15.43:0/1496336891 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2348020070 con 0x7f235806b710 2023-06-28T20:07:46.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.006+0000 7f235cafd700 1 -- 172.21.15.43:0/1496336891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23580b09b0 con 0x7f235806b710 2023-06-28T20:07:46.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.006+0000 7f2346ffd700 1 -- 172.21.15.43:0/1496336891 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23480257d0 con 0x7f235806b710 2023-06-28T20:07:46.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.006+0000 7f235cafd700 1 -- 172.21.15.43:0/1496336891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23580ca400 con 0x7f235806b710 2023-06-28T20:07:46.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.007+0000 7f235cafd700 1 -- 172.21.15.43:0/1496336891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23580b0ed0 con 0x7f235806b710 2023-06-28T20:07:46.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.008+0000 7f2346ffd700 1 -- 172.21.15.43:0/1496336891 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2348022030 con 0x7f235806b710 2023-06-28T20:07:46.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.008+0000 7f2346ffd700 1 --2- 172.21.15.43:0/1496336891 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f233806ace0 0x7f233806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:46.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.008+0000 7f2346ffd700 1 -- 172.21.15.43:0/1496336891 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f234808aa10 con 0x7f235806b710 2023-06-28T20:07:46.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.008+0000 7f2355d9b700 1 --2- 172.21.15.43:0/1496336891 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f233806ace0 0x7f233806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:46.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.009+0000 7f2355d9b700 1 --2- 172.21.15.43:0/1496336891 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f233806ace0 0x7f233806d190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f235806ecd0 tx=0x7f234c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:46.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.013+0000 7f2346ffd700 1 -- 172.21.15.43:0/1496336891 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f23580b0ed0 con 0x7f235806b710 2023-06-28T20:07:46.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.212+0000 7f235cafd700 1 -- 172.21.15.43:0/1496336891 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23580b0ed0 con 0x7f235806b710 2023-06-28T20:07:46.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.213+0000 7f2346ffd700 1 -- 172.21.15.43:0/1496336891 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18221 (secure 0 0 0) 0x7f23580b0ed0 con 0x7f235806b710 2023-06-28T20:07:46.217 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:46.217 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:07:22.377248+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:07:18.624759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-28T20:07:19.760017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:07:46.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.215+0000 7f235cafd700 1 -- 172.21.15.43:0/1496336891 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f233806ace0 msgr2=0x7f233806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:46.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.215+0000 7f235cafd700 1 --2- 172.21.15.43:0/1496336891 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f233806ace0 0x7f233806d190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f235806ecd0 tx=0x7f234c009040 comp rx=0 tx=0).stop 2023-06-28T20:07:46.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.215+0000 7f235cafd700 1 -- 172.21.15.43:0/1496336891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235806b710 msgr2=0x7f23580c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:46.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.215+0000 7f235cafd700 1 --2- 172.21.15.43:0/1496336891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235806b710 0x7f23580c7db0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2348013040 tx=0x7f2348004750 comp rx=0 tx=0).stop 2023-06-28T20:07:46.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.215+0000 7f235cafd700 1 -- 172.21.15.43:0/1496336891 shutdown_connections 2023-06-28T20:07:46.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.215+0000 7f235cafd700 1 --2- 172.21.15.43:0/1496336891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23580d0630 0x7f23580c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.215+0000 7f235cafd700 1 --2- 172.21.15.43:0/1496336891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f235806b710 0x7f23580c7db0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.215+0000 7f235cafd700 1 --2- 172.21.15.43:0/1496336891 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f233806ace0 0x7f233806d190 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.215+0000 7f235cafd700 1 --2- 172.21.15.43:0/1496336891 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23580cbd40 0x7f23580c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.215+0000 7f235cafd700 1 -- 172.21.15.43:0/1496336891 >> 172.21.15.43:0/1496336891 conn(0x7f23580664b0 msgr2=0x7f23580ad930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:46.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.215+0000 7f235cafd700 1 -- 172.21.15.43:0/1496336891 shutdown_connections 2023-06-28T20:07:46.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.215+0000 7f235cafd700 1 -- 172.21.15.43:0/1496336891 wait complete. 2023-06-28T20:07:46.231 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-28T20:07:13.580969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-28T20:07:14.038358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-28T20:07:18.624759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-28T20:07:19.760017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-28T20:07:46.232 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-28T20:07:46.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.356+0000 7efd1d82a700 1 Processor -- start 2023-06-28T20:07:46.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.356+0000 7efd1d82a700 1 -- start start 2023-06-28T20:07:46.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.357+0000 7efd1d82a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd1806a830 0x7efd180c9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:46.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.357+0000 7efd1d82a700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efd1806b420 0x7efd1806c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:46.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.357+0000 7efd1d82a700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd180ce370 con 0x7efd1806ac00 2023-06-28T20:07:46.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.357+0000 7efd1d82a700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd1806b070 con 0x7efd1806b420 2023-06-28T20:07:46.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.357+0000 7efd1d82a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1806b8a0 con 0x7efd1806a830 2023-06-28T20:07:46.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.358+0000 7efd177fe700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efd1806b420 0x7efd1806c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:46.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.358+0000 7efd16ffd700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd1806a830 0x7efd180c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:46.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.358+0000 7efd177fe700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efd1806b420 0x7efd1806c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:39808/0 (socket says 172.21.15.43:39808) 2023-06-28T20:07:46.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.358+0000 7efd167fc700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7efd1806ac00 0x7efd18069ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:42050/0 (socket says 172.21.15.43:42050) 2023-06-28T20:07:46.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.358+0000 7efd177fe700 1 -- 172.21.15.43:0/2991302486 learned_addr learned my addr 172.21.15.43:0/2991302486 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:46.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.358+0000 7efd177fe700 1 -- 172.21.15.43:0/2991302486 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd1806a830 msgr2=0x7efd180c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:46.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.358+0000 7efd177fe700 1 --2- 172.21.15.43:0/2991302486 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd1806a830 0x7efd180c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.358+0000 7efd177fe700 1 -- 172.21.15.43:0/2991302486 >> v1:172.21.15.178:6789/0 conn(0x7efd1806ac00 legacy=0x7efd18069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:46.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.358+0000 7efd177fe700 1 -- 172.21.15.43:0/2991302486 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd180ce610 con 0x7efd1806b420 2023-06-28T20:07:46.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.358+0000 7efd16ffd700 1 --2- 172.21.15.43:0/2991302486 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd1806a830 0x7efd180c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:46.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.358+0000 7efd177fe700 1 --2- 172.21.15.43:0/2991302486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd1806b420 0x7efd1806c0a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efd1000af60 tx=0x7efd1000af90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2a15740124cb31a server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:46.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.358+0000 7efd15ffb700 1 -- 172.21.15.43:0/2991302486 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1000f070 con 0x7efd1806b420 2023-06-28T20:07:46.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.358+0000 7efd15ffb700 1 -- 172.21.15.43:0/2991302486 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1000a2e0 con 0x7efd1806b420 2023-06-28T20:07:46.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.358+0000 7efd15ffb700 1 -- 172.21.15.43:0/2991302486 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1000cc00 con 0x7efd1806b420 2023-06-28T20:07:46.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.359+0000 7efd1d82a700 1 -- 172.21.15.43:0/2991302486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd1806b420 msgr2=0x7efd1806c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:46.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.359+0000 7efd1d82a700 1 --2- 172.21.15.43:0/2991302486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd1806b420 0x7efd1806c0a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efd1000af60 tx=0x7efd1000af90 comp rx=0 tx=0).stop 2023-06-28T20:07:46.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.359+0000 7efd1d82a700 1 -- 172.21.15.43:0/2991302486 shutdown_connections 2023-06-28T20:07:46.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.359+0000 7efd1d82a700 1 --2- 172.21.15.43:0/2991302486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd1806b420 0x7efd1806c0a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.359+0000 7efd1d82a700 1 --2- 172.21.15.43:0/2991302486 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd1806a830 0x7efd180c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.359+0000 7efd1d82a700 1 -- 172.21.15.43:0/2991302486 >> 172.21.15.43:0/2991302486 conn(0x7efd180b1a50 msgr2=0x7efd180cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:46.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.359+0000 7efd1d82a700 1 -- 172.21.15.43:0/2991302486 shutdown_connections 2023-06-28T20:07:46.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.359+0000 7efd1d82a700 1 -- 172.21.15.43:0/2991302486 wait complete. 2023-06-28T20:07:46.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.360+0000 7efd1d82a700 1 Processor -- start 2023-06-28T20:07:46.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.360+0000 7efd1d82a700 1 -- start start 2023-06-28T20:07:46.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.361+0000 7efd1d82a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd1806a830 0x7efd1814a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:46.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.361+0000 7efd1d82a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd1806ac00 0x7efd180c3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.361+0000 7efd1d82a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd1806b420 0x7efd1815b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.361+0000 7efd1d82a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd181f1350 con 0x7efd1806ac00 2023-06-28T20:07:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.361+0000 7efd1d82a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd181f1630 con 0x7efd1806b420 2023-06-28T20:07:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.361+0000 7efd1d82a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd181f1910 con 0x7efd1806a830 2023-06-28T20:07:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.361+0000 7efd167fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd1806ac00 0x7efd180c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:46.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.361+0000 7efd167fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd1806ac00 0x7efd180c3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53748/0 (socket says 172.21.15.43:53748) 2023-06-28T20:07:46.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.361+0000 7efd167fc700 1 -- 172.21.15.43:0/1586701449 learned_addr learned my addr 172.21.15.43:0/1586701449 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:46.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.361+0000 7efd16ffd700 1 --2- 172.21.15.43:0/1586701449 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd1806a830 0x7efd1814a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:46.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.361+0000 7efd177fe700 1 --2- 172.21.15.43:0/1586701449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd1806b420 0x7efd1815b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:46.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.362+0000 7efd167fc700 1 -- 172.21.15.43:0/1586701449 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd1806a830 msgr2=0x7efd1814a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:46.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.362+0000 7efd167fc700 1 --2- 172.21.15.43:0/1586701449 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd1806a830 0x7efd1814a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.362+0000 7efd167fc700 1 -- 172.21.15.43:0/1586701449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd1806b420 msgr2=0x7efd1815b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:46.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.362+0000 7efd167fc700 1 --2- 172.21.15.43:0/1586701449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd1806b420 0x7efd1815b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.362+0000 7efd167fc700 1 -- 172.21.15.43:0/1586701449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0c0094a0 con 0x7efd1806ac00 2023-06-28T20:07:46.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.362+0000 7efd167fc700 1 --2- 172.21.15.43:0/1586701449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd1806ac00 0x7efd180c3280 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efd0800c520 tx=0x7efd0800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:46.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.362+0000 7efcfffff700 1 -- 172.21.15.43:0/1586701449 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd08005cb0 con 0x7efd1806ac00 2023-06-28T20:07:46.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.362+0000 7efcfffff700 1 -- 172.21.15.43:0/1586701449 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd08005e10 con 0x7efd1806ac00 2023-06-28T20:07:46.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.362+0000 7efd1d82a700 1 -- 172.21.15.43:0/1586701449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1000ac40 con 0x7efd1806ac00 2023-06-28T20:07:46.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.362+0000 7efcfffff700 1 -- 172.21.15.43:0/1586701449 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd080072d0 con 0x7efd1806ac00 2023-06-28T20:07:46.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.362+0000 7efd1d82a700 1 -- 172.21.15.43:0/1586701449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd181f1ed0 con 0x7efd1806ac00 2023-06-28T20:07:46.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.363+0000 7efd1d82a700 1 -- 172.21.15.43:0/1586701449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd180728b0 con 0x7efd1806ac00 2023-06-28T20:07:46.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.364+0000 7efcfffff700 1 -- 172.21.15.43:0/1586701449 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efd08007430 con 0x7efd1806ac00 2023-06-28T20:07:46.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.364+0000 7efcfffff700 1 --2- 172.21.15.43:0/1586701449 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efd0406ace0 0x7efd0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:46.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.364+0000 7efd16ffd700 1 --2- 172.21.15.43:0/1586701449 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efd0406ace0 0x7efd0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:46.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.364+0000 7efcfffff700 1 -- 172.21.15.43:0/1586701449 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efd08088680 con 0x7efd1806ac00 2023-06-28T20:07:46.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.365+0000 7efd16ffd700 1 --2- 172.21.15.43:0/1586701449 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efd0406ace0 0x7efd0406d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efd180b0b70 tx=0x7efd0c009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:46.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.369+0000 7efcfffff700 1 -- 172.21.15.43:0/1586701449 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efd08053870 con 0x7efd1806ac00 2023-06-28T20:07:46.566 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.563+0000 7efd1d82a700 1 -- 172.21.15.43:0/1586701449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efd18071580 con 0x7efd1806ac00 2023-06-28T20:07:46.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.564+0000 7efcfffff700 1 -- 172.21.15.43:0/1586701449 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7efd08056e90 con 0x7efd1806ac00 2023-06-28T20:07:46.567 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-06-28T20:07:46.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.566+0000 7efd1d82a700 1 -- 172.21.15.43:0/1586701449 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efd0406ace0 msgr2=0x7efd0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:46.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.566+0000 7efd1d82a700 1 --2- 172.21.15.43:0/1586701449 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efd0406ace0 0x7efd0406d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efd180b0b70 tx=0x7efd0c009210 comp rx=0 tx=0).stop 2023-06-28T20:07:46.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.566+0000 7efd1d82a700 1 -- 172.21.15.43:0/1586701449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd1806ac00 msgr2=0x7efd180c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:46.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.566+0000 7efd1d82a700 1 --2- 172.21.15.43:0/1586701449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd1806ac00 0x7efd180c3280 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7efd0800c520 tx=0x7efd0800c830 comp rx=0 tx=0).stop 2023-06-28T20:07:46.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.566+0000 7efd1d82a700 1 -- 172.21.15.43:0/1586701449 shutdown_connections 2023-06-28T20:07:46.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.566+0000 7efd1d82a700 1 --2- 172.21.15.43:0/1586701449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd1806b420 0x7efd1815b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.566+0000 7efd1d82a700 1 --2- 172.21.15.43:0/1586701449 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efd0406ace0 0x7efd0406d190 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.566+0000 7efd1d82a700 1 --2- 172.21.15.43:0/1586701449 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efd1806a830 0x7efd1814a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.566+0000 7efd1d82a700 1 --2- 172.21.15.43:0/1586701449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efd1806ac00 0x7efd180c3280 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.566+0000 7efd1d82a700 1 -- 172.21.15.43:0/1586701449 >> 172.21.15.43:0/1586701449 conn(0x7efd180b1a50 msgr2=0x7efd180cd240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:46.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.566+0000 7efd1d82a700 1 -- 172.21.15.43:0/1586701449 shutdown_connections 2023-06-28T20:07:46.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.566+0000 7efd1d82a700 1 -- 172.21.15.43:0/1586701449 wait complete. 2023-06-28T20:07:46.581 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-28T20:07:46.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.705+0000 7f13ec9f4700 1 Processor -- start 2023-06-28T20:07:46.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.705+0000 7f13ec9f4700 1 -- start start 2023-06-28T20:07:46.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.706+0000 7f13ec9f4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f13e815a5f0 0x7f13e80ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:46.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.706+0000 7f13ec9f4700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13e80ad7d0 0x7f13e80ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:46.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.706+0000 7f13ec9f4700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e80ae950 con 0x7f13e80aede0 2023-06-28T20:07:46.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.706+0000 7f13ec9f4700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13e815a9c0 con 0x7f13e815a5f0 2023-06-28T20:07:46.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.706+0000 7f13ec9f4700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e80add40 con 0x7f13e80ad7d0 2023-06-28T20:07:46.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.706+0000 7f13e659c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f13e815a5f0 0x7f13e80ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:46.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.707+0000 7f13e659c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f13e815a5f0 0x7f13e80ce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53754/0 (socket says 172.21.15.43:53754) 2023-06-28T20:07:46.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.707+0000 7f13e659c700 1 -- 172.21.15.43:0/3234453383 learned_addr learned my addr 172.21.15.43:0/3234453383 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:46.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.707+0000 7f13e5d9b700 1 --2- 172.21.15.43:0/3234453383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13e80ad7d0 0x7f13e80ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:46.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.707+0000 7f13e659c700 1 -- 172.21.15.43:0/3234453383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13e80ad7d0 msgr2=0x7f13e80ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:46.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.707+0000 7f13e659c700 1 --2- 172.21.15.43:0/3234453383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13e80ad7d0 0x7f13e80ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.707+0000 7f13e659c700 1 -- 172.21.15.43:0/3234453383 >> v1:172.21.15.178:6789/0 conn(0x7f13e80aede0 legacy=0x7f13e80af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:46.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.707+0000 7f13e659c700 1 -- 172.21.15.43:0/3234453383 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e80ae020 con 0x7f13e815a5f0 2023-06-28T20:07:46.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.707+0000 7f13e659c700 1 --2- 172.21.15.43:0/3234453383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e815a5f0 0x7f13e80ce7d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f13dc006fd0 tx=0x7f13dc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc6d9bbd2e8f32c server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:46.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.707+0000 7f13e559a700 1 -- 172.21.15.43:0/3234453383 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc005e30 con 0x7f13e815a5f0 2023-06-28T20:07:46.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.707+0000 7f13e559a700 1 -- 172.21.15.43:0/3234453383 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13dc004030 con 0x7f13e815a5f0 2023-06-28T20:07:46.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.707+0000 7f13e559a700 1 -- 172.21.15.43:0/3234453383 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc0043e0 con 0x7f13e815a5f0 2023-06-28T20:07:46.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.708+0000 7f13ec9f4700 1 -- 172.21.15.43:0/3234453383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e815a5f0 msgr2=0x7f13e80ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:46.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.708+0000 7f13ec9f4700 1 --2- 172.21.15.43:0/3234453383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e815a5f0 0x7f13e80ce7d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f13dc006fd0 tx=0x7f13dc009bd0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.708+0000 7f13ec9f4700 1 -- 172.21.15.43:0/3234453383 shutdown_connections 2023-06-28T20:07:46.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.708+0000 7f13ec9f4700 1 --2- 172.21.15.43:0/3234453383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e815a5f0 0x7f13e80ce7d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.708+0000 7f13ec9f4700 1 --2- 172.21.15.43:0/3234453383 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13e80ad7d0 0x7f13e80ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.708+0000 7f13ec9f4700 1 -- 172.21.15.43:0/3234453383 >> 172.21.15.43:0/3234453383 conn(0x7f13e80664b0 msgr2=0x7f13e80d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:46.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.708+0000 7f13ec9f4700 1 -- 172.21.15.43:0/3234453383 shutdown_connections 2023-06-28T20:07:46.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.708+0000 7f13ec9f4700 1 -- 172.21.15.43:0/3234453383 wait complete. 2023-06-28T20:07:46.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.709+0000 7f13ec9f4700 1 Processor -- start 2023-06-28T20:07:46.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.709+0000 7f13ec9f4700 1 -- start start 2023-06-28T20:07:46.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.710+0000 7f13ec9f4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e80ad7d0 0x7f13e806c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:46.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.710+0000 7f13ec9f4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13e80aede0 0x7f13e8150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:46.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.710+0000 7f13ec9f4700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13e815a5f0 0x7f13e80b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:46.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.710+0000 7f13ec9f4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e806c6d0 con 0x7f13e80ad7d0 2023-06-28T20:07:46.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.710+0000 7f13ec9f4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e80cce60 con 0x7f13e80aede0 2023-06-28T20:07:46.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.710+0000 7f13ec9f4700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e80cd140 con 0x7f13e815a5f0 2023-06-28T20:07:46.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.710+0000 7f13e659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e80ad7d0 0x7f13e806c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:46.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.710+0000 7f13e659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e80ad7d0 0x7f13e806c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53760/0 (socket says 172.21.15.43:53760) 2023-06-28T20:07:46.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.710+0000 7f13e659c700 1 -- 172.21.15.43:0/2212932481 learned_addr learned my addr 172.21.15.43:0/2212932481 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:46.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.710+0000 7f13e5d9b700 1 --2- 172.21.15.43:0/2212932481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13e80aede0 0x7f13e8150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:46.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.710+0000 7f13e6d9d700 1 --2- 172.21.15.43:0/2212932481 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13e815a5f0 0x7f13e80b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:46.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.710+0000 7f13e659c700 1 -- 172.21.15.43:0/2212932481 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13e815a5f0 msgr2=0x7f13e80b2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:46.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.710+0000 7f13e659c700 1 --2- 172.21.15.43:0/2212932481 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13e815a5f0 0x7f13e80b2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.710+0000 7f13e659c700 1 -- 172.21.15.43:0/2212932481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13e80aede0 msgr2=0x7f13e8150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:46.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.710+0000 7f13e659c700 1 --2- 172.21.15.43:0/2212932481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13e80aede0 0x7f13e8150a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.710+0000 7f13e659c700 1 -- 172.21.15.43:0/2212932481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13dc009500 con 0x7f13e80ad7d0 2023-06-28T20:07:46.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.710+0000 7f13e5d9b700 1 --2- 172.21.15.43:0/2212932481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13e80aede0 0x7f13e8150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:46.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.711+0000 7f13e659c700 1 --2- 172.21.15.43:0/2212932481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e80ad7d0 0x7f13e806c190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f13dc009c10 tx=0x7f13dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:46.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.711+0000 7f13d6ffd700 1 -- 172.21.15.43:0/2212932481 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc003dd0 con 0x7f13e80ad7d0 2023-06-28T20:07:46.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.711+0000 7f13d6ffd700 1 -- 172.21.15.43:0/2212932481 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13dc01c420 con 0x7f13e80ad7d0 2023-06-28T20:07:46.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.711+0000 7f13ec9f4700 1 -- 172.21.15.43:0/2212932481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e80cd420 con 0x7f13e80ad7d0 2023-06-28T20:07:46.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.711+0000 7f13d6ffd700 1 -- 172.21.15.43:0/2212932481 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc01c7d0 con 0x7f13e80ad7d0 2023-06-28T20:07:46.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.711+0000 7f13ec9f4700 1 -- 172.21.15.43:0/2212932481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e80cd9e0 con 0x7f13e80ad7d0 2023-06-28T20:07:46.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.712+0000 7f13ec9f4700 1 -- 172.21.15.43:0/2212932481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e80cdb70 con 0x7f13e80ad7d0 2023-06-28T20:07:46.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.712+0000 7f13d6ffd700 1 -- 172.21.15.43:0/2212932481 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f13dc01c930 con 0x7f13e80ad7d0 2023-06-28T20:07:46.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.713+0000 7f13d6ffd700 1 --2- 172.21.15.43:0/2212932481 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f13c806ace0 0x7f13c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:46.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.713+0000 7f13d6ffd700 1 -- 172.21.15.43:0/2212932481 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f13dc08a4c0 con 0x7f13e80ad7d0 2023-06-28T20:07:46.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.713+0000 7f13e5d9b700 1 --2- 172.21.15.43:0/2212932481 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f13c806ace0 0x7f13c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:46.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.713+0000 7f13e5d9b700 1 --2- 172.21.15.43:0/2212932481 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f13c806ace0 0x7f13c806d190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f13e80cddc0 tx=0x7f13d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:46.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.717+0000 7f13d6ffd700 1 -- 172.21.15.43:0/2212932481 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f13dc055600 con 0x7f13e80ad7d0 2023-06-28T20:07:46.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.916+0000 7f13ec9f4700 1 -- 172.21.15.43:0/2212932481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f13e80728b0 con 0x7f13e80ad7d0 2023-06-28T20:07:46.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.916+0000 7f13d6ffd700 1 -- 172.21.15.43:0/2212932481 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f13dc058c20 con 0x7f13e80ad7d0 2023-06-28T20:07:46.920 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-06-28T20:07:46.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.918+0000 7f13ec9f4700 1 -- 172.21.15.43:0/2212932481 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f13c806ace0 msgr2=0x7f13c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:46.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.918+0000 7f13ec9f4700 1 --2- 172.21.15.43:0/2212932481 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f13c806ace0 0x7f13c806d190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f13e80cddc0 tx=0x7f13d8008040 comp rx=0 tx=0).stop 2023-06-28T20:07:46.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.918+0000 7f13ec9f4700 1 -- 172.21.15.43:0/2212932481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e80ad7d0 msgr2=0x7f13e806c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:46.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.918+0000 7f13ec9f4700 1 --2- 172.21.15.43:0/2212932481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e80ad7d0 0x7f13e806c190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f13dc009c10 tx=0x7f13dc004750 comp rx=0 tx=0).stop 2023-06-28T20:07:46.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.918+0000 7f13ec9f4700 1 -- 172.21.15.43:0/2212932481 shutdown_connections 2023-06-28T20:07:46.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.918+0000 7f13ec9f4700 1 --2- 172.21.15.43:0/2212932481 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f13c806ace0 0x7f13c806d190 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.918+0000 7f13ec9f4700 1 --2- 172.21.15.43:0/2212932481 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f13e815a5f0 0x7f13e80b2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.918+0000 7f13ec9f4700 1 --2- 172.21.15.43:0/2212932481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e80ad7d0 0x7f13e806c190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.918+0000 7f13ec9f4700 1 --2- 172.21.15.43:0/2212932481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13e80aede0 0x7f13e8150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:46.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.919+0000 7f13ec9f4700 1 -- 172.21.15.43:0/2212932481 >> 172.21.15.43:0/2212932481 conn(0x7f13e80664b0 msgr2=0x7f13e80b6370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:46.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.919+0000 7f13ec9f4700 1 -- 172.21.15.43:0/2212932481 shutdown_connections 2023-06-28T20:07:46.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:46.919+0000 7f13ec9f4700 1 -- 172.21.15.43:0/2212932481 wait complete. 2023-06-28T20:07:46.934 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-28T20:07:47.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.059+0000 7f6405b7a700 1 Processor -- start 2023-06-28T20:07:47.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.059+0000 7f6405b7a700 1 -- start start 2023-06-28T20:07:47.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.059+0000 7f6405b7a700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f64000af060 0x7f6400160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:47.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.060+0000 7f6405b7a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f640006ccc0 0x7f64000b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:47.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.060+0000 7f6405b7a700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6400160250 con 0x7f64000aec90 2023-06-28T20:07:47.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.060+0000 7f6405b7a700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f640015cb30 con 0x7f64000af060 2023-06-28T20:07:47.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.060+0000 7f6405b7a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6400071410 con 0x7f640006ccc0 2023-06-28T20:07:47.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.060+0000 7f63feffd700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f64000af060 0x7f6400160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:47.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.060+0000 7f63fffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f640006ccc0 0x7f64000b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:47.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.060+0000 7f63feffd700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f64000af060 0x7f6400160cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:39834/0 (socket says 172.21.15.43:39834) 2023-06-28T20:07:47.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.060+0000 7f63ff7fe700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f64000aec90 0x7f6400070bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:42078/0 (socket says 172.21.15.43:42078) 2023-06-28T20:07:47.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.060+0000 7f63fffff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f640006ccc0 0x7f64000b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.43:37744/0 (socket says 172.21.15.43:37744) 2023-06-28T20:07:47.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.060+0000 7f63feffd700 1 -- 172.21.15.43:0/434408822 learned_addr learned my addr 172.21.15.43:0/434408822 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:47.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.060+0000 7f63feffd700 1 -- 172.21.15.43:0/434408822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f640006ccc0 msgr2=0x7f64000b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:47.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.060+0000 7f63feffd700 1 --2- 172.21.15.43:0/434408822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f640006ccc0 0x7f64000b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.060+0000 7f63feffd700 1 -- 172.21.15.43:0/434408822 >> v1:172.21.15.178:6789/0 conn(0x7f64000aec90 legacy=0x7f6400070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:47.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.060+0000 7f63feffd700 1 -- 172.21.15.43:0/434408822 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64001604f0 con 0x7f64000af060 2023-06-28T20:07:47.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.060+0000 7f63fffff700 1 --2- 172.21.15.43:0/434408822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f640006ccc0 0x7f64000b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:47.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.060+0000 7f63feffd700 1 --2- 172.21.15.43:0/434408822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64000af060 0x7f6400160cf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f63f40098f0 tx=0x7f63f4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3913e671aeadb0cd server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:47.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.061+0000 7f63fe7fc700 1 -- 172.21.15.43:0/434408822 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f4011830 con 0x7f64000af060 2023-06-28T20:07:47.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.061+0000 7f63fe7fc700 1 -- 172.21.15.43:0/434408822 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f4011990 con 0x7f64000af060 2023-06-28T20:07:47.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.061+0000 7f63fe7fc700 1 -- 172.21.15.43:0/434408822 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f4011d20 con 0x7f64000af060 2023-06-28T20:07:47.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.061+0000 7f6405b7a700 1 -- 172.21.15.43:0/434408822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64000af060 msgr2=0x7f6400160cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:47.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.061+0000 7f6405b7a700 1 --2- 172.21.15.43:0/434408822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64000af060 0x7f6400160cf0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f63f40098f0 tx=0x7f63f4009c00 comp rx=0 tx=0).stop 2023-06-28T20:07:47.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.061+0000 7f6405b7a700 1 -- 172.21.15.43:0/434408822 shutdown_connections 2023-06-28T20:07:47.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.061+0000 7f6405b7a700 1 --2- 172.21.15.43:0/434408822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f640006ccc0 0x7f64000b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.061+0000 7f6405b7a700 1 --2- 172.21.15.43:0/434408822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64000af060 0x7f6400160cf0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.061+0000 7f6405b7a700 1 -- 172.21.15.43:0/434408822 >> 172.21.15.43:0/434408822 conn(0x7f640006b000 msgr2=0x7f640006b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:47.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.061+0000 7f6405b7a700 1 -- 172.21.15.43:0/434408822 shutdown_connections 2023-06-28T20:07:47.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.061+0000 7f6405b7a700 1 -- 172.21.15.43:0/434408822 wait complete. 2023-06-28T20:07:47.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.062+0000 7f6405b7a700 1 Processor -- start 2023-06-28T20:07:47.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.063+0000 7f6405b7a700 1 -- start start 2023-06-28T20:07:47.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.063+0000 7f6405b7a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f640006ccc0 0x7f64000b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:47.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.063+0000 7f6405b7a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64000aec90 0x7f64000c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:47.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.063+0000 7f6405b7a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64000af060 0x7f6400156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:47.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.063+0000 7f6405b7a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64000b31d0 con 0x7f64000aec90 2023-06-28T20:07:47.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.063+0000 7f6405b7a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64000c9a30 con 0x7f640006ccc0 2023-06-28T20:07:47.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.063+0000 7f6405b7a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64000c6bc0 con 0x7f64000af060 2023-06-28T20:07:47.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.063+0000 7f63feffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64000aec90 0x7f64000c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:47.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.063+0000 7f63feffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64000aec90 0x7f64000c94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53776/0 (socket says 172.21.15.43:53776) 2023-06-28T20:07:47.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.063+0000 7f63feffd700 1 -- 172.21.15.43:0/2959324615 learned_addr learned my addr 172.21.15.43:0/2959324615 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:47.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.063+0000 7f63fffff700 1 --2- 172.21.15.43:0/2959324615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64000af060 0x7f6400156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:47.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.064+0000 7f63ff7fe700 1 --2- 172.21.15.43:0/2959324615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f640006ccc0 0x7f64000b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:47.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.064+0000 7f63feffd700 1 -- 172.21.15.43:0/2959324615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64000af060 msgr2=0x7f6400156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:47.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.064+0000 7f63feffd700 1 --2- 172.21.15.43:0/2959324615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64000af060 0x7f6400156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.064+0000 7f63feffd700 1 -- 172.21.15.43:0/2959324615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f640006ccc0 msgr2=0x7f64000b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:47.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.064+0000 7f63feffd700 1 --2- 172.21.15.43:0/2959324615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f640006ccc0 0x7f64000b2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.064+0000 7f63feffd700 1 -- 172.21.15.43:0/2959324615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f8009a70 con 0x7f64000aec90 2023-06-28T20:07:47.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.064+0000 7f63feffd700 1 --2- 172.21.15.43:0/2959324615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64000aec90 0x7f64000c94f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009c40 tx=0x7f63f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:47.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.064+0000 7f63e7fff700 1 -- 172.21.15.43:0/2959324615 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f40044d0 con 0x7f64000aec90 2023-06-28T20:07:47.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.064+0000 7f63e7fff700 1 -- 172.21.15.43:0/2959324615 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f401e070 con 0x7f64000aec90 2023-06-28T20:07:47.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.064+0000 7f6405b7a700 1 -- 172.21.15.43:0/2959324615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f4009550 con 0x7f64000aec90 2023-06-28T20:07:47.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.064+0000 7f63e7fff700 1 -- 172.21.15.43:0/2959324615 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f40038c0 con 0x7f64000aec90 2023-06-28T20:07:47.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.064+0000 7f6405b7a700 1 -- 172.21.15.43:0/2959324615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64000c7160 con 0x7f64000aec90 2023-06-28T20:07:47.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.065+0000 7f6405b7a700 1 -- 172.21.15.43:0/2959324615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64000728b0 con 0x7f64000aec90 2023-06-28T20:07:47.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.066+0000 7f63e7fff700 1 -- 172.21.15.43:0/2959324615 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f63f4003a20 con 0x7f64000aec90 2023-06-28T20:07:47.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.066+0000 7f63e7fff700 1 --2- 172.21.15.43:0/2959324615 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f63ec06ace0 0x7f63ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:47.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.066+0000 7f63e7fff700 1 -- 172.21.15.43:0/2959324615 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f63f408a220 con 0x7f64000aec90 2023-06-28T20:07:47.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.066+0000 7f63ff7fe700 1 --2- 172.21.15.43:0/2959324615 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f63ec06ace0 0x7f63ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:47.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.066+0000 7f63ff7fe700 1 --2- 172.21.15.43:0/2959324615 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f63ec06ace0 0x7f63ec06d190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f63f0000f60 tx=0x7f63f0007ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:47.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.070+0000 7f63e7fff700 1 -- 172.21.15.43:0/2959324615 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f63f4055360 con 0x7f64000aec90 2023-06-28T20:07:47.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.267+0000 7f6405b7a700 1 -- 172.21.15.43:0/2959324615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f640006d9c0 con 0x7f64000aec90 2023-06-28T20:07:47.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.267+0000 7f63e7fff700 1 -- 172.21.15.43:0/2959324615 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f63f402c3b0 con 0x7f64000aec90 2023-06-28T20:07:47.270 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-06-28T20:07:47.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.269+0000 7f6405b7a700 1 -- 172.21.15.43:0/2959324615 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f63ec06ace0 msgr2=0x7f63ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:47.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.269+0000 7f6405b7a700 1 --2- 172.21.15.43:0/2959324615 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f63ec06ace0 0x7f63ec06d190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f63f0000f60 tx=0x7f63f0007ed0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.269+0000 7f6405b7a700 1 -- 172.21.15.43:0/2959324615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64000aec90 msgr2=0x7f64000c94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.269+0000 7f6405b7a700 1 --2- 172.21.15.43:0/2959324615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64000aec90 0x7f64000c94f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009c40 tx=0x7f63f4004750 comp rx=0 tx=0).stop 2023-06-28T20:07:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.269+0000 7f6405b7a700 1 -- 172.21.15.43:0/2959324615 shutdown_connections 2023-06-28T20:07:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.269+0000 7f6405b7a700 1 --2- 172.21.15.43:0/2959324615 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f63ec06ace0 0x7f63ec06d190 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.269+0000 7f6405b7a700 1 --2- 172.21.15.43:0/2959324615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f64000af060 0x7f6400156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.269+0000 7f6405b7a700 1 --2- 172.21.15.43:0/2959324615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f640006ccc0 0x7f64000b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.269+0000 7f6405b7a700 1 --2- 172.21.15.43:0/2959324615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64000aec90 0x7f64000c94f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.269+0000 7f6405b7a700 1 -- 172.21.15.43:0/2959324615 >> 172.21.15.43:0/2959324615 conn(0x7f640006b000 msgr2=0x7f640006d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.269+0000 7f6405b7a700 1 -- 172.21.15.43:0/2959324615 shutdown_connections 2023-06-28T20:07:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.269+0000 7f6405b7a700 1 -- 172.21.15.43:0/2959324615 wait complete. 2023-06-28T20:07:47.284 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-28T20:07:47.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.410+0000 7f9cf9882700 1 Processor -- start 2023-06-28T20:07:47.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.410+0000 7f9cf9882700 1 -- start start 2023-06-28T20:07:47.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.410+0000 7f9cf9882700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9cf40cf830 0x7f9cf40cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:47.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.411+0000 7f9cf9882700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9cf415cf10 0x7f9cf40b82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:47.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.411+0000 7f9cf9882700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf40d0f00 con 0x7f9cf40d0a80 2023-06-28T20:07:47.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.411+0000 7f9cf9882700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf40d0140 con 0x7f9cf415cf10 2023-06-28T20:07:47.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.411+0000 7f9cf9882700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf40b8c80 con 0x7f9cf40cf830 2023-06-28T20:07:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.411+0000 7f9cf27fc700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9cf40d0a80 0x7f9cf40d1c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42970/0 (socket says 172.21.15.43:42970) 2023-06-28T20:07:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.411+0000 7f9cf27fc700 1 -- 172.21.15.43:0/4139258113 learned_addr learned my addr 172.21.15.43:0/4139258113 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.411+0000 7f9cf37fe700 1 --2- 172.21.15.43:0/4139258113 >> v2:172.21.15.43:3300/0 conn(0x7f9cf415cf10 0x7f9cf40b82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.411+0000 7f9cf2ffd700 1 --2- 172.21.15.43:0/4139258113 >> v2:172.21.15.178:3300/0 conn(0x7f9cf40cf830 0x7f9cf40cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.411+0000 7f9cf1ffb700 1 -- 172.21.15.43:0/4139258113 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 459685526 0 0) 0x7f9cf40d0f00 con 0x7f9cf40d0a80 2023-06-28T20:07:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.411+0000 7f9cf1ffb700 1 -- 172.21.15.43:0/4139258113 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cdc0035b0 con 0x7f9cf40d0a80 2023-06-28T20:07:47.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.411+0000 7f9cf37fe700 1 -- 172.21.15.43:0/4139258113 >> v2:172.21.15.178:3300/0 conn(0x7f9cf40cf830 msgr2=0x7f9cf40cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.411+0000 7f9cf37fe700 1 --2- 172.21.15.43:0/4139258113 >> v2:172.21.15.178:3300/0 conn(0x7f9cf40cf830 0x7f9cf40cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.411+0000 7f9cf37fe700 1 -- 172.21.15.43:0/4139258113 >> v1:172.21.15.43:6789/0 conn(0x7f9cf40d0a80 legacy=0x7f9cf40d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.411+0000 7f9cf37fe700 1 -- 172.21.15.43:0/4139258113 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf40d11a0 con 0x7f9cf415cf10 2023-06-28T20:07:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.411+0000 7f9cf37fe700 1 --2- 172.21.15.43:0/4139258113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf415cf10 0x7f9cf40b82e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9cf40cf800 tx=0x7f9cec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd00f7a0a124653 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.412+0000 7f9cf1ffb700 1 -- 172.21.15.43:0/4139258113 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec00e070 con 0x7f9cf415cf10 2023-06-28T20:07:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.412+0000 7f9cf1ffb700 1 -- 172.21.15.43:0/4139258113 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cec00bb20 con 0x7f9cf415cf10 2023-06-28T20:07:47.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.412+0000 7f9cf1ffb700 1 -- 172.21.15.43:0/4139258113 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec012430 con 0x7f9cf415cf10 2023-06-28T20:07:47.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.412+0000 7f9cf9882700 1 -- 172.21.15.43:0/4139258113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf415cf10 msgr2=0x7f9cf40b82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:47.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.412+0000 7f9cf9882700 1 --2- 172.21.15.43:0/4139258113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf415cf10 0x7f9cf40b82e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9cf40cf800 tx=0x7f9cec00b330 comp rx=0 tx=0).stop 2023-06-28T20:07:47.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.412+0000 7f9cf9882700 1 -- 172.21.15.43:0/4139258113 shutdown_connections 2023-06-28T20:07:47.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.412+0000 7f9cf9882700 1 --2- 172.21.15.43:0/4139258113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf415cf10 0x7f9cf40b82e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.412+0000 7f9cf9882700 1 --2- 172.21.15.43:0/4139258113 >> v2:172.21.15.178:3300/0 conn(0x7f9cf40cf830 0x7f9cf40cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.412+0000 7f9cf9882700 1 -- 172.21.15.43:0/4139258113 >> 172.21.15.43:0/4139258113 conn(0x7f9cf406a640 msgr2=0x7f9cf406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:47.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.412+0000 7f9cf9882700 1 -- 172.21.15.43:0/4139258113 shutdown_connections 2023-06-28T20:07:47.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.412+0000 7f9cf9882700 1 -- 172.21.15.43:0/4139258113 wait complete. 2023-06-28T20:07:47.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.413+0000 7f9cf9882700 1 Processor -- start 2023-06-28T20:07:47.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.414+0000 7f9cf9882700 1 -- start start 2023-06-28T20:07:47.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.414+0000 7f9cf9882700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cf40cf830 0x7f9cf40b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:47.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.414+0000 7f9cf9882700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9cf40d0a80 0x7f9cf40c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.414+0000 7f9cf9882700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf415cf10 0x7f9cf415ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.414+0000 7f9cf9882700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf4155650 con 0x7f9cf415cf10 2023-06-28T20:07:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.414+0000 7f9cf9882700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf40b5410 con 0x7f9cf40cf830 2023-06-28T20:07:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.414+0000 7f9cf9882700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf41549d0 con 0x7f9cf40d0a80 2023-06-28T20:07:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.414+0000 7f9cf37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf415cf10 0x7f9cf415ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.414+0000 7f9cf37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf415cf10 0x7f9cf415ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53792/0 (socket says 172.21.15.43:53792) 2023-06-28T20:07:47.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.414+0000 7f9cf37fe700 1 -- 172.21.15.43:0/1434196147 learned_addr learned my addr 172.21.15.43:0/1434196147 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.415+0000 7f9cf2ffd700 1 --2- 172.21.15.43:0/1434196147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cf40cf830 0x7f9cf40b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.415+0000 7f9cf27fc700 1 --2- 172.21.15.43:0/1434196147 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9cf40d0a80 0x7f9cf40c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.415+0000 7f9cf37fe700 1 -- 172.21.15.43:0/1434196147 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9cf40d0a80 msgr2=0x7f9cf40c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.415+0000 7f9cf37fe700 1 --2- 172.21.15.43:0/1434196147 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9cf40d0a80 0x7f9cf40c7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.415+0000 7f9cf37fe700 1 -- 172.21.15.43:0/1434196147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cf40cf830 msgr2=0x7f9cf40b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.415+0000 7f9cf37fe700 1 --2- 172.21.15.43:0/1434196147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cf40cf830 0x7f9cf40b4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.415+0000 7f9cf37fe700 1 -- 172.21.15.43:0/1434196147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec009a70 con 0x7f9cf415cf10 2023-06-28T20:07:47.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.415+0000 7f9cf27fc700 1 --2- 172.21.15.43:0/1434196147 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9cf40d0a80 0x7f9cf40c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:47.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.415+0000 7f9cf2ffd700 1 --2- 172.21.15.43:0/1434196147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cf40cf830 0x7f9cf40b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:47.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.415+0000 7f9cf37fe700 1 --2- 172.21.15.43:0/1434196147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf415cf10 0x7f9cf415ed00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9cec014040 tx=0x7f9cec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:47.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.415+0000 7f9cdbfff700 1 -- 172.21.15.43:0/1434196147 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec012430 con 0x7f9cf415cf10 2023-06-28T20:07:47.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.415+0000 7f9cdbfff700 1 -- 172.21.15.43:0/1434196147 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cec020070 con 0x7f9cf415cf10 2023-06-28T20:07:47.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.415+0000 7f9cf9882700 1 -- 172.21.15.43:0/1434196147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf4154cb0 con 0x7f9cf415cf10 2023-06-28T20:07:47.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.415+0000 7f9cdbfff700 1 -- 172.21.15.43:0/1434196147 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec004280 con 0x7f9cf415cf10 2023-06-28T20:07:47.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.415+0000 7f9cf9882700 1 -- 172.21.15.43:0/1434196147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf4155270 con 0x7f9cf415cf10 2023-06-28T20:07:47.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.416+0000 7f9cf9882700 1 -- 172.21.15.43:0/1434196147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf4155790 con 0x7f9cf415cf10 2023-06-28T20:07:47.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.417+0000 7f9cdbfff700 1 -- 172.21.15.43:0/1434196147 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9cec0043e0 con 0x7f9cf415cf10 2023-06-28T20:07:47.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.417+0000 7f9cdbfff700 1 --2- 172.21.15.43:0/1434196147 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9ce006ace0 0x7f9ce006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:47.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.417+0000 7f9cdbfff700 1 -- 172.21.15.43:0/1434196147 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9cec022070 con 0x7f9cf415cf10 2023-06-28T20:07:47.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.417+0000 7f9cf2ffd700 1 --2- 172.21.15.43:0/1434196147 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9ce006ace0 0x7f9ce006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:47.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.417+0000 7f9cf2ffd700 1 --2- 172.21.15.43:0/1434196147 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9ce006ace0 0x7f9ce006d190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4009c30 tx=0x7f9ce4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:47.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.421+0000 7f9cdbfff700 1 -- 172.21.15.43:0/1434196147 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9cf4155790 con 0x7f9cf415cf10 2023-06-28T20:07:47.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.617+0000 7f9cf9882700 1 -- 172.21.15.43:0/1434196147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9cf4155790 con 0x7f9cf415cf10 2023-06-28T20:07:47.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.617+0000 7f9cdbfff700 1 -- 172.21.15.43:0/1434196147 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9cf4155790 con 0x7f9cf415cf10 2023-06-28T20:07:47.621 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-06-28T20:07:47.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.619+0000 7f9cf9882700 1 -- 172.21.15.43:0/1434196147 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9ce006ace0 msgr2=0x7f9ce006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:47.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.619+0000 7f9cf9882700 1 --2- 172.21.15.43:0/1434196147 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9ce006ace0 0x7f9ce006d190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4009c30 tx=0x7f9ce4008040 comp rx=0 tx=0).stop 2023-06-28T20:07:47.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.619+0000 7f9cf9882700 1 -- 172.21.15.43:0/1434196147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf415cf10 msgr2=0x7f9cf415ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:47.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.619+0000 7f9cf9882700 1 --2- 172.21.15.43:0/1434196147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf415cf10 0x7f9cf415ed00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9cec014040 tx=0x7f9cec004ad0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.619+0000 7f9cf9882700 1 -- 172.21.15.43:0/1434196147 shutdown_connections 2023-06-28T20:07:47.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.619+0000 7f9cf9882700 1 --2- 172.21.15.43:0/1434196147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cf415cf10 0x7f9cf415ed00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.619+0000 7f9cf9882700 1 --2- 172.21.15.43:0/1434196147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cf40cf830 0x7f9cf40b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.619+0000 7f9cf9882700 1 --2- 172.21.15.43:0/1434196147 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9ce006ace0 0x7f9ce006d190 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.619+0000 7f9cf9882700 1 --2- 172.21.15.43:0/1434196147 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9cf40d0a80 0x7f9cf40c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.619+0000 7f9cf9882700 1 -- 172.21.15.43:0/1434196147 >> 172.21.15.43:0/1434196147 conn(0x7f9cf406a640 msgr2=0x7f9cf40ce300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:47.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.619+0000 7f9cf9882700 1 -- 172.21.15.43:0/1434196147 shutdown_connections 2023-06-28T20:07:47.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.620+0000 7f9cf9882700 1 -- 172.21.15.43:0/1434196147 wait complete. 2023-06-28T20:07:47.635 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:07:47.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.755+0000 7fb147890700 1 Processor -- start 2023-06-28T20:07:47.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.756+0000 7fb147890700 1 -- start start 2023-06-28T20:07:47.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.756+0000 7fb147890700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb1400cefc0 0x7fb1400cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:47.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.757+0000 7fb147890700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb14015ab50 con 0x7fb1400cb8e0 2023-06-28T20:07:47.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.757+0000 7fb147890700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1400cd1b0 con 0x7fb1400d10f0 2023-06-28T20:07:47.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.757+0000 7fb147890700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1400cc110 con 0x7fb1400cefc0 2023-06-28T20:07:47.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.757+0000 7fb144e2b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb1400cefc0 0x7fb1400cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:47.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.757+0000 7fb145e2d700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb1400cb8e0 0x7fb1400b0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42986/0 (socket says 172.21.15.43:42986) 2023-06-28T20:07:47.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.757+0000 7fb145e2d700 1 -- 172.21.15.43:0/132171823 learned_addr learned my addr 172.21.15.43:0/132171823 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:47.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.757+0000 7fb144e2b700 1 -- 172.21.15.43:0/132171823 >> v1:172.21.15.178:6789/0 conn(0x7fb1400d10f0 legacy=0x7fb1400ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:47.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.757+0000 7fb144e2b700 1 -- 172.21.15.43:0/132171823 >> v1:172.21.15.43:6789/0 conn(0x7fb1400cb8e0 legacy=0x7fb1400b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:47.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.757+0000 7fb133fff700 1 -- 172.21.15.43:0/132171823 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2271675777 0 0) 0x7fb14015ab50 con 0x7fb1400cb8e0 2023-06-28T20:07:47.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.757+0000 7fb144e2b700 1 -- 172.21.15.43:0/132171823 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1400cc3f0 con 0x7fb1400cefc0 2023-06-28T20:07:47.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.757+0000 7fb144e2b700 1 --2- 172.21.15.43:0/132171823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1400cefc0 0x7fb1400cb3a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb134009840 tx=0x7fb134009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612563f736f7f64a server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:47.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.757+0000 7fb133fff700 1 -- 172.21.15.43:0/132171823 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134011830 con 0x7fb1400cefc0 2023-06-28T20:07:47.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.757+0000 7fb133fff700 1 -- 172.21.15.43:0/132171823 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb134011990 con 0x7fb1400cefc0 2023-06-28T20:07:47.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.757+0000 7fb133fff700 1 -- 172.21.15.43:0/132171823 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134011d20 con 0x7fb1400cefc0 2023-06-28T20:07:47.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.758+0000 7fb147890700 1 -- 172.21.15.43:0/132171823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1400cefc0 msgr2=0x7fb1400cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:47.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.758+0000 7fb147890700 1 --2- 172.21.15.43:0/132171823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1400cefc0 0x7fb1400cb3a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb134009840 tx=0x7fb134009b50 comp rx=0 tx=0).stop 2023-06-28T20:07:47.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.758+0000 7fb147890700 1 -- 172.21.15.43:0/132171823 shutdown_connections 2023-06-28T20:07:47.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.758+0000 7fb147890700 1 --2- 172.21.15.43:0/132171823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1400cefc0 0x7fb1400cb3a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.758+0000 7fb147890700 1 -- 172.21.15.43:0/132171823 >> 172.21.15.43:0/132171823 conn(0x7fb1400664b0 msgr2=0x7fb14009a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:47.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.758+0000 7fb147890700 1 -- 172.21.15.43:0/132171823 shutdown_connections 2023-06-28T20:07:47.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.758+0000 7fb147890700 1 -- 172.21.15.43:0/132171823 wait complete. 2023-06-28T20:07:47.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.760+0000 7fb147890700 1 Processor -- start 2023-06-28T20:07:47.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.760+0000 7fb147890700 1 -- start start 2023-06-28T20:07:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.761+0000 7fb147890700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1400cb8e0 0x7fb1401f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.761+0000 7fb147890700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1400cefc0 0x7fb1400b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.761+0000 7fb147890700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1400d10f0 0x7fb14014bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.761+0000 7fb147890700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1400b5e30 con 0x7fb1400d10f0 2023-06-28T20:07:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.761+0000 7fb147890700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1400b60d0 con 0x7fb1400cefc0 2023-06-28T20:07:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.761+0000 7fb147890700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1400b63b0 con 0x7fb1400cb8e0 2023-06-28T20:07:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.761+0000 7fb145e2d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1400d10f0 0x7fb14014bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.761+0000 7fb145e2d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1400d10f0 0x7fb14014bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53806/0 (socket says 172.21.15.43:53806) 2023-06-28T20:07:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.761+0000 7fb145e2d700 1 -- 172.21.15.43:0/2892588931 learned_addr learned my addr 172.21.15.43:0/2892588931 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.761+0000 7fb144e2b700 1 --2- 172.21.15.43:0/2892588931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1400cefc0 0x7fb1400b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.761+0000 7fb14562c700 1 --2- 172.21.15.43:0/2892588931 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1400cb8e0 0x7fb1401f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.761+0000 7fb145e2d700 1 -- 172.21.15.43:0/2892588931 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1400cb8e0 msgr2=0x7fb1401f3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.761+0000 7fb145e2d700 1 --2- 172.21.15.43:0/2892588931 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1400cb8e0 0x7fb1401f3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.761+0000 7fb145e2d700 1 -- 172.21.15.43:0/2892588931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1400cefc0 msgr2=0x7fb1400b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.761+0000 7fb145e2d700 1 --2- 172.21.15.43:0/2892588931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1400cefc0 0x7fb1400b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.761+0000 7fb145e2d700 1 -- 172.21.15.43:0/2892588931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1340094a0 con 0x7fb1400d10f0 2023-06-28T20:07:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.761+0000 7fb144e2b700 1 --2- 172.21.15.43:0/2892588931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1400cefc0 0x7fb1400b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.761+0000 7fb145e2d700 1 --2- 172.21.15.43:0/2892588931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1400d10f0 0x7fb14014bda0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb13800cca0 tx=0x7fb13800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.761+0000 7fb131ffb700 1 -- 172.21.15.43:0/2892588931 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13800cea0 con 0x7fb1400d10f0 2023-06-28T20:07:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.761+0000 7fb131ffb700 1 -- 172.21.15.43:0/2892588931 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13800ade0 con 0x7fb1400d10f0 2023-06-28T20:07:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.761+0000 7fb147890700 1 -- 172.21.15.43:0/2892588931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1400b6690 con 0x7fb1400d10f0 2023-06-28T20:07:47.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.761+0000 7fb131ffb700 1 -- 172.21.15.43:0/2892588931 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb138007310 con 0x7fb1400d10f0 2023-06-28T20:07:47.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.762+0000 7fb147890700 1 -- 172.21.15.43:0/2892588931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14014c2e0 con 0x7fb1400d10f0 2023-06-28T20:07:47.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.762+0000 7fb147890700 1 -- 172.21.15.43:0/2892588931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1401e1f60 con 0x7fb1400d10f0 2023-06-28T20:07:47.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.763+0000 7fb131ffb700 1 -- 172.21.15.43:0/2892588931 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb138007470 con 0x7fb1400d10f0 2023-06-28T20:07:47.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.763+0000 7fb131ffb700 1 --2- 172.21.15.43:0/2892588931 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb12406ad10 0x7fb12406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:47.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.763+0000 7fb131ffb700 1 -- 172.21.15.43:0/2892588931 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb138089020 con 0x7fb1400d10f0 2023-06-28T20:07:47.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.763+0000 7fb14562c700 1 --2- 172.21.15.43:0/2892588931 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb12406ad10 0x7fb12406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:47.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.764+0000 7fb14562c700 1 --2- 172.21.15.43:0/2892588931 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb12406ad10 0x7fb12406d1c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb13c000fb0 tx=0x7fb13c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:47.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.768+0000 7fb131ffb700 1 -- 172.21.15.43:0/2892588931 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb1401e1f60 con 0x7fb1400d10f0 2023-06-28T20:07:47.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.966+0000 7fb147890700 1 -- 172.21.15.43:0/2892588931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1401e1f60 con 0x7fb1400d10f0 2023-06-28T20:07:47.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.967+0000 7fb131ffb700 1 -- 172.21.15.43:0/2892588931 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18221 (secure 0 0 0) 0x7fb1401e1f60 con 0x7fb1400d10f0 2023-06-28T20:07:47.970 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:47.970 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:07:22.377248+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:07:18.624759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-28T20:07:19.760017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:07:47.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.969+0000 7fb147890700 1 -- 172.21.15.43:0/2892588931 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb12406ad10 msgr2=0x7fb12406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:47.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.969+0000 7fb147890700 1 --2- 172.21.15.43:0/2892588931 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb12406ad10 0x7fb12406d1c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb13c000fb0 tx=0x7fb13c009040 comp rx=0 tx=0).stop 2023-06-28T20:07:47.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.969+0000 7fb147890700 1 -- 172.21.15.43:0/2892588931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1400d10f0 msgr2=0x7fb14014bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:47.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.969+0000 7fb147890700 1 --2- 172.21.15.43:0/2892588931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1400d10f0 0x7fb14014bda0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb13800cca0 tx=0x7fb13800a340 comp rx=0 tx=0).stop 2023-06-28T20:07:47.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.969+0000 7fb147890700 1 -- 172.21.15.43:0/2892588931 shutdown_connections 2023-06-28T20:07:47.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.969+0000 7fb147890700 1 --2- 172.21.15.43:0/2892588931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1400d10f0 0x7fb14014bda0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.969+0000 7fb147890700 1 --2- 172.21.15.43:0/2892588931 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb12406ad10 0x7fb12406d1c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.969+0000 7fb147890700 1 --2- 172.21.15.43:0/2892588931 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb1400cb8e0 0x7fb1401f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.969+0000 7fb147890700 1 --2- 172.21.15.43:0/2892588931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1400cefc0 0x7fb1400b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:47.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.969+0000 7fb147890700 1 -- 172.21.15.43:0/2892588931 >> 172.21.15.43:0/2892588931 conn(0x7fb1400664b0 msgr2=0x7fb1400d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:47.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.969+0000 7fb147890700 1 -- 172.21.15.43:0/2892588931 shutdown_connections 2023-06-28T20:07:47.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:47.969+0000 7fb147890700 1 -- 172.21.15.43:0/2892588931 wait complete. 2023-06-28T20:07:47.985 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-28T20:07:13.580969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-28T20:07:14.038358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-28T20:07:18.624759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-28T20:07:19.760017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-28T20:07:47.985 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-28T20:07:48.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.112+0000 7f64a0cad700 1 Processor -- start 2023-06-28T20:07:48.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.112+0000 7f64a0cad700 1 -- start start 2023-06-28T20:07:48.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.113+0000 7f64a0cad700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f649c0cf830 0x7f649c0cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:48.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.113+0000 7f64a0cad700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f649c15cf10 0x7f649c0b82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:48.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.113+0000 7f64a0cad700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f649c0d0f00 con 0x7f649c0d0a80 2023-06-28T20:07:48.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.113+0000 7f64a0cad700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f649c0d0140 con 0x7f649c0cf830 2023-06-28T20:07:48.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.113+0000 7f64a0cad700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649c0b8c80 con 0x7f649c15cf10 2023-06-28T20:07:48.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.113+0000 7f6499d9b700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f649c0d0a80 0x7f649c0d1c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42998/0 (socket says 172.21.15.43:42998) 2023-06-28T20:07:48.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.113+0000 7f6499d9b700 1 -- 172.21.15.43:0/2208204371 learned_addr learned my addr 172.21.15.43:0/2208204371 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:48.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.113+0000 7f649ad9d700 1 --2- 172.21.15.43:0/2208204371 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f649c15cf10 0x7f649c0b82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:48.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.113+0000 7f649a59c700 1 --2- 172.21.15.43:0/2208204371 >> v2:172.21.15.178:3300/0 conn(0x7f649c0cf830 0x7f649c0cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:48.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.113+0000 7f649959a700 1 -- 172.21.15.43:0/2208204371 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3915300794 0 0) 0x7f649c0d0f00 con 0x7f649c0d0a80 2023-06-28T20:07:48.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.113+0000 7f649959a700 1 -- 172.21.15.43:0/2208204371 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64840035b0 con 0x7f649c0d0a80 2023-06-28T20:07:48.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.114+0000 7f649959a700 1 -- 172.21.15.43:0/2208204371 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1228205595 0 0) 0x7f64840035b0 con 0x7f649c0d0a80 2023-06-28T20:07:48.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.114+0000 7f649959a700 1 -- 172.21.15.43:0/2208204371 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f649c15cf10 msgr2=0x7f649c0b82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:48.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.114+0000 7f649959a700 1 --2- 172.21.15.43:0/2208204371 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f649c15cf10 0x7f649c0b82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.114+0000 7f649959a700 1 -- 172.21.15.43:0/2208204371 >> v2:172.21.15.178:3300/0 conn(0x7f649c0cf830 msgr2=0x7f649c0cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:48.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.114+0000 7f649959a700 1 --2- 172.21.15.43:0/2208204371 >> v2:172.21.15.178:3300/0 conn(0x7f649c0cf830 0x7f649c0cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.114+0000 7f649959a700 1 -- 172.21.15.43:0/2208204371 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f649c0d11a0 con 0x7f649c0d0a80 2023-06-28T20:07:48.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.114+0000 7f649959a700 1 -- 172.21.15.43:0/2208204371 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f6490001670 con 0x7f649c0d0a80 2023-06-28T20:07:48.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.114+0000 7f649959a700 1 -- 172.21.15.43:0/2208204371 >> v1:172.21.15.43:6789/0 conn(0x7f649c0d0a80 legacy=0x7f649c0d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:48.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.114+0000 7f649959a700 1 --2- 172.21.15.43:0/2208204371 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f649c15cf10 0x7f6484003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:48.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.114+0000 7f649959a700 1 --2- 172.21.15.43:0/2208204371 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6484003f80 0x7f6484004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:48.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.114+0000 7f649959a700 1 --2- 172.21.15.43:0/2208204371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6484004d90 0x7f6484005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:48.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.114+0000 7f649959a700 1 -- 172.21.15.43:0/2208204371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649c0d0140 con 0x7f6484004d90 2023-06-28T20:07:48.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.114+0000 7f649959a700 1 -- 172.21.15.43:0/2208204371 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649c0b8c80 con 0x7f6484003f80 2023-06-28T20:07:48.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.114+0000 7f649959a700 1 -- 172.21.15.43:0/2208204371 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f648400ba10 con 0x7f649c15cf10 2023-06-28T20:07:48.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.114+0000 7f649ad9d700 1 --2- 172.21.15.43:0/2208204371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6484004d90 0x7f6484005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:48.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.114+0000 7f649a59c700 1 --2- 172.21.15.43:0/2208204371 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f649c15cf10 0x7f6484003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:48.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.114+0000 7f6499d9b700 1 --2- 172.21.15.43:0/2208204371 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6484003f80 0x7f6484004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:48.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.114+0000 7f649ad9d700 1 -- 172.21.15.43:0/2208204371 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f649c15cf10 msgr2=0x7f6484003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:48.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.114+0000 7f649ad9d700 1 --2- 172.21.15.43:0/2208204371 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f649c15cf10 0x7f6484003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.114+0000 7f649ad9d700 1 -- 172.21.15.43:0/2208204371 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6484003f80 msgr2=0x7f6484004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:48.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.114+0000 7f649ad9d700 1 --2- 172.21.15.43:0/2208204371 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6484003f80 0x7f6484004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.114+0000 7f649ad9d700 1 -- 172.21.15.43:0/2208204371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64840035b0 con 0x7f6484004d90 2023-06-28T20:07:48.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.114+0000 7f649ad9d700 1 --2- 172.21.15.43:0/2208204371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6484004d90 0x7f6484005200 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f649c0d03f0 tx=0x7f6484005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df855f415c4f8333 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:48.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.115+0000 7f649959a700 1 -- 172.21.15.43:0/2208204371 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c00ebd0 con 0x7f6484004d90 2023-06-28T20:07:48.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.115+0000 7f649959a700 1 -- 172.21.15.43:0/2208204371 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648c00f040 con 0x7f6484004d90 2023-06-28T20:07:48.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.115+0000 7f649959a700 1 -- 172.21.15.43:0/2208204371 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c014480 con 0x7f6484004d90 2023-06-28T20:07:48.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.115+0000 7f64a0cad700 1 -- 172.21.15.43:0/2208204371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6484004d90 msgr2=0x7f6484005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:48.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.115+0000 7f64a0cad700 1 --2- 172.21.15.43:0/2208204371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6484004d90 0x7f6484005200 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f649c0d03f0 tx=0x7f6484005880 comp rx=0 tx=0).stop 2023-06-28T20:07:48.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.115+0000 7f649ad9d700 1 -- 172.21.15.43:0/2208204371 reap_dead start 2023-06-28T20:07:48.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.115+0000 7f64a0cad700 1 -- 172.21.15.43:0/2208204371 shutdown_connections 2023-06-28T20:07:48.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.115+0000 7f64a0cad700 1 -- 172.21.15.43:0/2208204371 >> 172.21.15.43:0/2208204371 conn(0x7f649c06a640 msgr2=0x7f649c06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:48.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.115+0000 7f64a0cad700 1 -- 172.21.15.43:0/2208204371 shutdown_connections 2023-06-28T20:07:48.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.115+0000 7f64a0cad700 1 -- 172.21.15.43:0/2208204371 wait complete. 2023-06-28T20:07:48.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.117+0000 7f64a0cad700 1 Processor -- start 2023-06-28T20:07:48.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.117+0000 7f64a0cad700 1 -- start start 2023-06-28T20:07:48.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.117+0000 7f64a0cad700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6484004d90 0x7f649c15a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:48.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.117+0000 7f64a0cad700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649c15ef30 0x7f649c0b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:48.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.117+0000 7f64a0cad700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f649c0b4d70 0x7f649c1f24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:48.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.117+0000 7f64a0cad700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649c0b5210 con 0x7f649c15ef30 2023-06-28T20:07:48.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.117+0000 7f64a0cad700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649c15de90 con 0x7f649c0b4d70 2023-06-28T20:07:48.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.117+0000 7f64a0cad700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f649c15e170 con 0x7f6484004d90 2023-06-28T20:07:48.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.117+0000 7f6499d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649c15ef30 0x7f649c0b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:48.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.118+0000 7f6499d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649c15ef30 0x7f649c0b4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53824/0 (socket says 172.21.15.43:53824) 2023-06-28T20:07:48.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.118+0000 7f6499d9b700 1 -- 172.21.15.43:0/1954805281 learned_addr learned my addr 172.21.15.43:0/1954805281 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:48.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.118+0000 7f649a59c700 1 --2- 172.21.15.43:0/1954805281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6484004d90 0x7f649c15a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:48.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.118+0000 7f649ad9d700 1 --2- 172.21.15.43:0/1954805281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f649c0b4d70 0x7f649c1f24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:48.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.118+0000 7f6499d9b700 1 -- 172.21.15.43:0/1954805281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6484004d90 msgr2=0x7f649c15a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:48.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.118+0000 7f6499d9b700 1 --2- 172.21.15.43:0/1954805281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6484004d90 0x7f649c15a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.118+0000 7f6499d9b700 1 -- 172.21.15.43:0/1954805281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f649c0b4d70 msgr2=0x7f649c1f24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:48.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.118+0000 7f6499d9b700 1 --2- 172.21.15.43:0/1954805281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f649c0b4d70 0x7f649c1f24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.118+0000 7f6499d9b700 1 -- 172.21.15.43:0/1954805281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f649c15e450 con 0x7f649c15ef30 2023-06-28T20:07:48.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.118+0000 7f6499d9b700 1 --2- 172.21.15.43:0/1954805281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649c15ef30 0x7f649c0b4830 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6490001aa0 tx=0x7f64900075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:48.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.118+0000 7f648affd700 1 -- 172.21.15.43:0/1954805281 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649000f690 con 0x7f649c15ef30 2023-06-28T20:07:48.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.118+0000 7f648affd700 1 -- 172.21.15.43:0/1954805281 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649000f810 con 0x7f649c15ef30 2023-06-28T20:07:48.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.118+0000 7f64a0cad700 1 -- 172.21.15.43:0/1954805281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f649c15e730 con 0x7f649c15ef30 2023-06-28T20:07:48.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.118+0000 7f648affd700 1 -- 172.21.15.43:0/1954805281 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649000fbc0 con 0x7f649c15ef30 2023-06-28T20:07:48.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.118+0000 7f64a0cad700 1 -- 172.21.15.43:0/1954805281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f649c15fe90 con 0x7f649c15ef30 2023-06-28T20:07:48.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.119+0000 7f64a0cad700 1 -- 172.21.15.43:0/1954805281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f649c15ec50 con 0x7f649c15ef30 2023-06-28T20:07:48.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.120+0000 7f648affd700 1 -- 172.21.15.43:0/1954805281 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f649000fd20 con 0x7f649c15ef30 2023-06-28T20:07:48.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.120+0000 7f648affd700 1 --2- 172.21.15.43:0/1954805281 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f647c06ace0 0x7f647c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:48.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.120+0000 7f648affd700 1 -- 172.21.15.43:0/1954805281 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f64900893f0 con 0x7f649c15ef30 2023-06-28T20:07:48.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.120+0000 7f649a59c700 1 --2- 172.21.15.43:0/1954805281 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f647c06ace0 0x7f647c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:48.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.120+0000 7f649a59c700 1 --2- 172.21.15.43:0/1954805281 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f647c06ace0 0x7f647c06d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6494009ce0 tx=0x7f6494009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:48.131 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.124+0000 7f648affd700 1 -- 172.21.15.43:0/1954805281 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f649c15ec50 con 0x7f649c15ef30 2023-06-28T20:07:48.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.321+0000 7f64a0cad700 1 -- 172.21.15.43:0/1954805281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f649c15ec50 con 0x7f649c15ef30 2023-06-28T20:07:48.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.321+0000 7f648affd700 1 -- 172.21.15.43:0/1954805281 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f649c15ec50 con 0x7f649c15ef30 2023-06-28T20:07:48.325 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-06-28T20:07:48.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.323+0000 7f64a0cad700 1 -- 172.21.15.43:0/1954805281 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f647c06ace0 msgr2=0x7f647c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:48.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.323+0000 7f64a0cad700 1 --2- 172.21.15.43:0/1954805281 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f647c06ace0 0x7f647c06d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6494009ce0 tx=0x7f6494009210 comp rx=0 tx=0).stop 2023-06-28T20:07:48.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.323+0000 7f64a0cad700 1 -- 172.21.15.43:0/1954805281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649c15ef30 msgr2=0x7f649c0b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:48.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.323+0000 7f64a0cad700 1 --2- 172.21.15.43:0/1954805281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649c15ef30 0x7f649c0b4830 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6490001aa0 tx=0x7f64900075f0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.324+0000 7f64a0cad700 1 -- 172.21.15.43:0/1954805281 shutdown_connections 2023-06-28T20:07:48.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.324+0000 7f64a0cad700 1 --2- 172.21.15.43:0/1954805281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f649c0b4d70 0x7f649c1f24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.324+0000 7f64a0cad700 1 --2- 172.21.15.43:0/1954805281 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f647c06ace0 0x7f647c06d190 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.324+0000 7f64a0cad700 1 --2- 172.21.15.43:0/1954805281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6484004d90 0x7f649c15a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.324+0000 7f64a0cad700 1 --2- 172.21.15.43:0/1954805281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f649c15ef30 0x7f649c0b4830 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.324+0000 7f64a0cad700 1 -- 172.21.15.43:0/1954805281 >> 172.21.15.43:0/1954805281 conn(0x7f649c06a640 msgr2=0x7f649c0cd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:48.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.324+0000 7f64a0cad700 1 -- 172.21.15.43:0/1954805281 shutdown_connections 2023-06-28T20:07:48.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.324+0000 7f64a0cad700 1 -- 172.21.15.43:0/1954805281 wait complete. 2023-06-28T20:07:48.339 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-28T20:07:48.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.464+0000 7f539980d700 1 Processor -- start 2023-06-28T20:07:48.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.464+0000 7f539980d700 1 -- start start 2023-06-28T20:07:48.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.465+0000 7f539980d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f53940cf830 0x7f53940cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:48.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.465+0000 7f539980d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53940d0a80 0x7f53940d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:48.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.465+0000 7f539980d700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53940b89f0 con 0x7f539415cf10 2023-06-28T20:07:48.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.465+0000 7f539980d700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53940d0f00 con 0x7f53940cf830 2023-06-28T20:07:48.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.465+0000 7f539980d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53940d0140 con 0x7f53940d0a80 2023-06-28T20:07:48.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.465+0000 7f5392ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f53940cf830 0x7f53940cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:48.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.465+0000 7f53937fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f539415cf10 0x7f53940b82e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43020/0 (socket says 172.21.15.43:43020) 2023-06-28T20:07:48.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.465+0000 7f53937fe700 1 -- 172.21.15.43:0/665861550 learned_addr learned my addr 172.21.15.43:0/665861550 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:48.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.465+0000 7f53927fc700 1 --2- 172.21.15.43:0/665861550 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53940d0a80 0x7f53940d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:48.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.465+0000 7f5392ffd700 1 -- 172.21.15.43:0/665861550 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53940d0a80 msgr2=0x7f53940d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:48.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.465+0000 7f5392ffd700 1 --2- 172.21.15.43:0/665861550 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53940d0a80 0x7f53940d1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.465+0000 7f5392ffd700 1 -- 172.21.15.43:0/665861550 >> v1:172.21.15.43:6789/0 conn(0x7f539415cf10 legacy=0x7f53940b82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:48.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.465+0000 7f5391ffb700 1 -- 172.21.15.43:0/665861550 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 796900040 0 0) 0x7f53940b89f0 con 0x7f539415cf10 2023-06-28T20:07:48.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.465+0000 7f5392ffd700 1 -- 172.21.15.43:0/665861550 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53940b8c80 con 0x7f53940cf830 2023-06-28T20:07:48.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.465+0000 7f5392ffd700 1 --2- 172.21.15.43:0/665861550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53940cf830 0x7f53940cfc00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5388009840 tx=0x7f5388009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d4225eaaeb0d63 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:48.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.466+0000 7f5391ffb700 1 -- 172.21.15.43:0/665861550 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5388011830 con 0x7f53940cf830 2023-06-28T20:07:48.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.466+0000 7f5391ffb700 1 -- 172.21.15.43:0/665861550 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5388011990 con 0x7f53940cf830 2023-06-28T20:07:48.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.466+0000 7f5391ffb700 1 -- 172.21.15.43:0/665861550 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5388011d20 con 0x7f53940cf830 2023-06-28T20:07:48.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.466+0000 7f539980d700 1 -- 172.21.15.43:0/665861550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53940cf830 msgr2=0x7f53940cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:48.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.466+0000 7f539980d700 1 --2- 172.21.15.43:0/665861550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53940cf830 0x7f53940cfc00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5388009840 tx=0x7f5388009b50 comp rx=0 tx=0).stop 2023-06-28T20:07:48.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.466+0000 7f539980d700 1 -- 172.21.15.43:0/665861550 shutdown_connections 2023-06-28T20:07:48.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.466+0000 7f539980d700 1 --2- 172.21.15.43:0/665861550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53940cf830 0x7f53940cfc00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.466+0000 7f539980d700 1 --2- 172.21.15.43:0/665861550 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53940d0a80 0x7f53940d1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.466+0000 7f539980d700 1 -- 172.21.15.43:0/665861550 >> 172.21.15.43:0/665861550 conn(0x7f539406a640 msgr2=0x7f539406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:48.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.466+0000 7f539980d700 1 -- 172.21.15.43:0/665861550 shutdown_connections 2023-06-28T20:07:48.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.466+0000 7f539980d700 1 -- 172.21.15.43:0/665861550 wait complete. 2023-06-28T20:07:48.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.467+0000 7f539980d700 1 Processor -- start 2023-06-28T20:07:48.475 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.468+0000 7f539980d700 1 -- start start 2023-06-28T20:07:48.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.468+0000 7f539980d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53940cf830 0x7f53940cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:48.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.468+0000 7f539980d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53940d0a80 0x7f53940c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:48.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.468+0000 7f539980d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f539415cf10 0x7f53940c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:48.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.468+0000 7f539980d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539415a380 con 0x7f539415cf10 2023-06-28T20:07:48.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.468+0000 7f539980d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53940ccd40 con 0x7f53940cf830 2023-06-28T20:07:48.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.468+0000 7f539980d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5394151e60 con 0x7f53940d0a80 2023-06-28T20:07:48.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.468+0000 7f53937fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f539415cf10 0x7f53940c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:48.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.468+0000 7f53937fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f539415cf10 0x7f53940c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53838/0 (socket says 172.21.15.43:53838) 2023-06-28T20:07:48.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.468+0000 7f53937fe700 1 -- 172.21.15.43:0/1103427748 learned_addr learned my addr 172.21.15.43:0/1103427748 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:48.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.469+0000 7f53927fc700 1 --2- 172.21.15.43:0/1103427748 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53940d0a80 0x7f53940c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:48.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.469+0000 7f5392ffd700 1 --2- 172.21.15.43:0/1103427748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53940cf830 0x7f53940cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:48.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.469+0000 7f53937fe700 1 -- 172.21.15.43:0/1103427748 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53940d0a80 msgr2=0x7f53940c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:48.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.469+0000 7f53937fe700 1 --2- 172.21.15.43:0/1103427748 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53940d0a80 0x7f53940c8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.469+0000 7f53937fe700 1 -- 172.21.15.43:0/1103427748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53940cf830 msgr2=0x7f53940cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:48.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.469+0000 7f53937fe700 1 --2- 172.21.15.43:0/1103427748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53940cf830 0x7f53940cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.469+0000 7f53937fe700 1 -- 172.21.15.43:0/1103427748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53880094a0 con 0x7f539415cf10 2023-06-28T20:07:48.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.469+0000 7f53927fc700 1 --2- 172.21.15.43:0/1103427748 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53940d0a80 0x7f53940c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:48.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.469+0000 7f53937fe700 1 --2- 172.21.15.43:0/1103427748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f539415cf10 0x7f53940c8780 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f538c00cc70 tx=0x7f538c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:48.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.469+0000 7f537bfff700 1 -- 172.21.15.43:0/1103427748 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c00ad80 con 0x7f539415cf10 2023-06-28T20:07:48.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.469+0000 7f537bfff700 1 -- 172.21.15.43:0/1103427748 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538c00ce70 con 0x7f539415cf10 2023-06-28T20:07:48.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.469+0000 7f537bfff700 1 -- 172.21.15.43:0/1103427748 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c0072f0 con 0x7f539415cf10 2023-06-28T20:07:48.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.469+0000 7f539980d700 1 -- 172.21.15.43:0/1103427748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5394159770 con 0x7f539415cf10 2023-06-28T20:07:48.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.469+0000 7f539980d700 1 -- 172.21.15.43:0/1103427748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5394159d30 con 0x7f539415cf10 2023-06-28T20:07:48.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.470+0000 7f539980d700 1 -- 172.21.15.43:0/1103427748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f539415a4f0 con 0x7f539415cf10 2023-06-28T20:07:48.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.470+0000 7f537bfff700 1 -- 172.21.15.43:0/1103427748 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f538c007450 con 0x7f539415cf10 2023-06-28T20:07:48.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.471+0000 7f537bfff700 1 --2- 172.21.15.43:0/1103427748 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f538006ace0 0x7f538006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:48.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.471+0000 7f537bfff700 1 -- 172.21.15.43:0/1103427748 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f538c089040 con 0x7f539415cf10 2023-06-28T20:07:48.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.471+0000 7f5392ffd700 1 --2- 172.21.15.43:0/1103427748 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f538006ace0 0x7f538006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:48.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.471+0000 7f5392ffd700 1 --2- 172.21.15.43:0/1103427748 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f538006ace0 0x7f538006d190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f53880090b0 tx=0x7f538801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:48.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.475+0000 7f537bfff700 1 -- 172.21.15.43:0/1103427748 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f538c0540f0 con 0x7f539415cf10 2023-06-28T20:07:48.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.675+0000 7f539980d700 1 -- 172.21.15.43:0/1103427748 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53940728b0 con 0x7f539415cf10 2023-06-28T20:07:48.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.675+0000 7f537bfff700 1 -- 172.21.15.43:0/1103427748 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f538c057710 con 0x7f539415cf10 2023-06-28T20:07:48.679 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-06-28T20:07:48.680 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.677+0000 7f539980d700 1 -- 172.21.15.43:0/1103427748 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f538006ace0 msgr2=0x7f538006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:48.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.677+0000 7f539980d700 1 --2- 172.21.15.43:0/1103427748 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f538006ace0 0x7f538006d190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f53880090b0 tx=0x7f538801c040 comp rx=0 tx=0).stop 2023-06-28T20:07:48.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.677+0000 7f539980d700 1 -- 172.21.15.43:0/1103427748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f539415cf10 msgr2=0x7f53940c8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:48.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.677+0000 7f539980d700 1 --2- 172.21.15.43:0/1103427748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f539415cf10 0x7f53940c8780 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f538c00cc70 tx=0x7f538c00a340 comp rx=0 tx=0).stop 2023-06-28T20:07:48.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.677+0000 7f539980d700 1 -- 172.21.15.43:0/1103427748 shutdown_connections 2023-06-28T20:07:48.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.677+0000 7f539980d700 1 --2- 172.21.15.43:0/1103427748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f539415cf10 0x7f53940c8780 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.681 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.677+0000 7f539980d700 1 --2- 172.21.15.43:0/1103427748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f53940cf830 0x7f53940cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.677+0000 7f539980d700 1 --2- 172.21.15.43:0/1103427748 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f538006ace0 0x7f538006d190 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.677+0000 7f539980d700 1 --2- 172.21.15.43:0/1103427748 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f53940d0a80 0x7f53940c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.677+0000 7f539980d700 1 -- 172.21.15.43:0/1103427748 >> 172.21.15.43:0/1103427748 conn(0x7f539406a640 msgr2=0x7f539415f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:48.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.678+0000 7f539980d700 1 -- 172.21.15.43:0/1103427748 shutdown_connections 2023-06-28T20:07:48.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.678+0000 7f539980d700 1 -- 172.21.15.43:0/1103427748 wait complete. 2023-06-28T20:07:48.693 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-28T20:07:48.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.817+0000 7f61e6349700 1 Processor -- start 2023-06-28T20:07:48.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.817+0000 7f61e6349700 1 -- start start 2023-06-28T20:07:48.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.818+0000 7f61e6349700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f61e015a770 0x7f61e00709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:48.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.818+0000 7f61e6349700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f61e00b1f90 0x7f61e0070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:48.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.818+0000 7f61e6349700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e015b540 con 0x7f61e015a3a0 2023-06-28T20:07:48.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.819+0000 7f61e6349700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e015ac10 con 0x7f61e00b1f90 2023-06-28T20:07:48.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.819+0000 7f61e6349700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e015aef0 con 0x7f61e015a770 2023-06-28T20:07:48.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.819+0000 7f61dffff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f61e015a3a0 0x7f61e00b1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43030/0 (socket says 172.21.15.43:43030) 2023-06-28T20:07:48.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.819+0000 7f61dffff700 1 -- 172.21.15.43:0/545065457 learned_addr learned my addr 172.21.15.43:0/545065457 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:48.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.819+0000 7f61e48e6700 1 --2- 172.21.15.43:0/545065457 >> v2:172.21.15.43:3300/0 conn(0x7f61e00b1f90 0x7f61e0070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:48.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.819+0000 7f61df7fe700 1 --2- 172.21.15.43:0/545065457 >> v2:172.21.15.178:3300/0 conn(0x7f61e015a770 0x7f61e00709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:48.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.819+0000 7f61deffd700 1 -- 172.21.15.43:0/545065457 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122131285 0 0) 0x7f61e015b540 con 0x7f61e015a3a0 2023-06-28T20:07:48.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.819+0000 7f61deffd700 1 -- 172.21.15.43:0/545065457 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61c80035b0 con 0x7f61e015a3a0 2023-06-28T20:07:48.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.819+0000 7f61e48e6700 1 -- 172.21.15.43:0/545065457 >> v2:172.21.15.178:3300/0 conn(0x7f61e015a770 msgr2=0x7f61e00709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:48.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.819+0000 7f61e48e6700 1 --2- 172.21.15.43:0/545065457 >> v2:172.21.15.178:3300/0 conn(0x7f61e015a770 0x7f61e00709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.819+0000 7f61e48e6700 1 -- 172.21.15.43:0/545065457 >> v1:172.21.15.43:6789/0 conn(0x7f61e015a3a0 legacy=0x7f61e00b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:48.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.819+0000 7f61e48e6700 1 -- 172.21.15.43:0/545065457 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e015b1d0 con 0x7f61e00b1f90 2023-06-28T20:07:48.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.819+0000 7f61e48e6700 1 --2- 172.21.15.43:0/545065457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61e00b1f90 0x7f61e0070ef0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f61e0070690 tx=0x7f61d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb591e2042302442 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:48.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.819+0000 7f61deffd700 1 -- 172.21.15.43:0/545065457 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d800e070 con 0x7f61e00b1f90 2023-06-28T20:07:48.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.819+0000 7f61deffd700 1 -- 172.21.15.43:0/545065457 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d800bb20 con 0x7f61e00b1f90 2023-06-28T20:07:48.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.819+0000 7f61deffd700 1 -- 172.21.15.43:0/545065457 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d8012430 con 0x7f61e00b1f90 2023-06-28T20:07:48.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.820+0000 7f61e6349700 1 -- 172.21.15.43:0/545065457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61e00b1f90 msgr2=0x7f61e0070ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.820+0000 7f61e6349700 1 --2- 172.21.15.43:0/545065457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61e00b1f90 0x7f61e0070ef0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f61e0070690 tx=0x7f61d800b330 comp rx=0 tx=0).stop 2023-06-28T20:07:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.820+0000 7f61e6349700 1 -- 172.21.15.43:0/545065457 shutdown_connections 2023-06-28T20:07:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.820+0000 7f61e6349700 1 --2- 172.21.15.43:0/545065457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61e00b1f90 0x7f61e0070ef0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.820+0000 7f61e6349700 1 --2- 172.21.15.43:0/545065457 >> v2:172.21.15.178:3300/0 conn(0x7f61e015a770 0x7f61e00709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.820+0000 7f61e6349700 1 -- 172.21.15.43:0/545065457 >> 172.21.15.43:0/545065457 conn(0x7f61e006b000 msgr2=0x7f61e006b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.820+0000 7f61e6349700 1 -- 172.21.15.43:0/545065457 shutdown_connections 2023-06-28T20:07:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.820+0000 7f61e6349700 1 -- 172.21.15.43:0/545065457 wait complete. 2023-06-28T20:07:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.821+0000 7f61e6349700 1 Processor -- start 2023-06-28T20:07:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.821+0000 7f61e6349700 1 -- start start 2023-06-28T20:07:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.822+0000 7f61e6349700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61e00b1f90 0x7f61e01ef910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.822+0000 7f61e6349700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61e015a3a0 0x7f61e0158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.822+0000 7f61e6349700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61e015a770 0x7f61e00b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.822+0000 7f61e6349700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e00b3660 con 0x7f61e015a770 2023-06-28T20:07:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.822+0000 7f61e6349700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e00b2a50 con 0x7f61e00b1f90 2023-06-28T20:07:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.822+0000 7f61e6349700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e00b2d30 con 0x7f61e015a3a0 2023-06-28T20:07:48.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.822+0000 7f61e48e6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61e015a770 0x7f61e00b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:48.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.822+0000 7f61e48e6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61e015a770 0x7f61e00b4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53852/0 (socket says 172.21.15.43:53852) 2023-06-28T20:07:48.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.822+0000 7f61e48e6700 1 -- 172.21.15.43:0/925562976 learned_addr learned my addr 172.21.15.43:0/925562976 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:48.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.822+0000 7f61dffff700 1 --2- 172.21.15.43:0/925562976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61e00b1f90 0x7f61e01ef910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:48.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.822+0000 7f61df7fe700 1 --2- 172.21.15.43:0/925562976 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61e015a3a0 0x7f61e0158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:48.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.823+0000 7f61e48e6700 1 -- 172.21.15.43:0/925562976 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61e015a3a0 msgr2=0x7f61e0158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:48.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.823+0000 7f61e48e6700 1 --2- 172.21.15.43:0/925562976 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61e015a3a0 0x7f61e0158db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.823+0000 7f61e48e6700 1 -- 172.21.15.43:0/925562976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61e00b1f90 msgr2=0x7f61e01ef910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:48.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.823+0000 7f61e48e6700 1 --2- 172.21.15.43:0/925562976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61e00b1f90 0x7f61e01ef910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:48.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.823+0000 7f61e48e6700 1 -- 172.21.15.43:0/925562976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d8009a70 con 0x7f61e015a770 2023-06-28T20:07:48.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.823+0000 7f61df7fe700 1 --2- 172.21.15.43:0/925562976 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61e015a3a0 0x7f61e0158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:48.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.823+0000 7f61e48e6700 1 --2- 172.21.15.43:0/925562976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61e015a770 0x7f61e00b4940 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f61d8014040 tx=0x7f61d8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:48.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.823+0000 7f61dcff9700 1 -- 172.21.15.43:0/925562976 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d8012430 con 0x7f61e015a770 2023-06-28T20:07:48.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.823+0000 7f61dcff9700 1 -- 172.21.15.43:0/925562976 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d8020070 con 0x7f61e015a770 2023-06-28T20:07:48.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.823+0000 7f61e6349700 1 -- 172.21.15.43:0/925562976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e00b3010 con 0x7f61e015a770 2023-06-28T20:07:48.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.823+0000 7f61dcff9700 1 -- 172.21.15.43:0/925562976 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61d8004280 con 0x7f61e015a770 2023-06-28T20:07:48.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.823+0000 7f61e6349700 1 -- 172.21.15.43:0/925562976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e014ed10 con 0x7f61e015a770 2023-06-28T20:07:48.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.824+0000 7f61e6349700 1 -- 172.21.15.43:0/925562976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e00728b0 con 0x7f61e015a770 2023-06-28T20:07:48.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.824+0000 7f61dcff9700 1 -- 172.21.15.43:0/925562976 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f61d80043e0 con 0x7f61e015a770 2023-06-28T20:07:48.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.825+0000 7f61dcff9700 1 --2- 172.21.15.43:0/925562976 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f61cc06ace0 0x7f61cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:48.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.825+0000 7f61dcff9700 1 -- 172.21.15.43:0/925562976 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f61d8022070 con 0x7f61e015a770 2023-06-28T20:07:48.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.825+0000 7f61dffff700 1 --2- 172.21.15.43:0/925562976 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f61cc06ace0 0x7f61cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:48.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.825+0000 7f61dffff700 1 --2- 172.21.15.43:0/925562976 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f61cc06ace0 0x7f61cc06d190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f61d40011e0 tx=0x7f61d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:48.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:48.829+0000 7f61dcff9700 1 -- 172.21.15.43:0/925562976 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f61d8056570 con 0x7f61e015a770 2023-06-28T20:07:49.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.028+0000 7f61e6349700 1 -- 172.21.15.43:0/925562976 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f61e00b37a0 con 0x7f61e015a770 2023-06-28T20:07:49.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.029+0000 7f61dcff9700 1 -- 172.21.15.43:0/925562976 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f61e00b37a0 con 0x7f61e015a770 2023-06-28T20:07:49.034 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-06-28T20:07:49.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.031+0000 7f61e6349700 1 -- 172.21.15.43:0/925562976 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f61cc06ace0 msgr2=0x7f61cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:49.035 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.031+0000 7f61e6349700 1 --2- 172.21.15.43:0/925562976 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f61cc06ace0 0x7f61cc06d190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f61d40011e0 tx=0x7f61d4009040 comp rx=0 tx=0).stop 2023-06-28T20:07:49.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.031+0000 7f61e6349700 1 -- 172.21.15.43:0/925562976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61e015a770 msgr2=0x7f61e00b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:49.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.031+0000 7f61e6349700 1 --2- 172.21.15.43:0/925562976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61e015a770 0x7f61e00b4940 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f61d8014040 tx=0x7f61d8004ad0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.031+0000 7f61e6349700 1 -- 172.21.15.43:0/925562976 shutdown_connections 2023-06-28T20:07:49.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.031+0000 7f61e6349700 1 --2- 172.21.15.43:0/925562976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61e015a770 0x7f61e00b4940 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.031+0000 7f61e6349700 1 --2- 172.21.15.43:0/925562976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61e00b1f90 0x7f61e01ef910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.031+0000 7f61e6349700 1 --2- 172.21.15.43:0/925562976 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f61cc06ace0 0x7f61cc06d190 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.031+0000 7f61e6349700 1 --2- 172.21.15.43:0/925562976 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f61e015a3a0 0x7f61e0158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.031+0000 7f61e6349700 1 -- 172.21.15.43:0/925562976 >> 172.21.15.43:0/925562976 conn(0x7f61e006b000 msgr2=0x7f61e00ccf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:49.037 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.031+0000 7f61e6349700 1 -- 172.21.15.43:0/925562976 shutdown_connections 2023-06-28T20:07:49.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.031+0000 7f61e6349700 1 -- 172.21.15.43:0/925562976 wait complete. 2023-06-28T20:07:49.046 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-28T20:07:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.172+0000 7fdc6ade3700 1 Processor -- start 2023-06-28T20:07:49.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.172+0000 7fdc6ade3700 1 -- start start 2023-06-28T20:07:49.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.172+0000 7fdc6ade3700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc640cf6e0 0x7fdc640ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:49.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.172+0000 7fdc6ade3700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc64160e90 con 0x7fdc640b1920 2023-06-28T20:07:49.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.172+0000 7fdc6ade3700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc64160bb0 con 0x7fdc640cca10 2023-06-28T20:07:49.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.172+0000 7fdc6ade3700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc641609c0 con 0x7fdc640cf6e0 2023-06-28T20:07:49.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.173+0000 7fdc69380700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdc640b1920 0x7fdc640b1d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43044/0 (socket says 172.21.15.43:43044) 2023-06-28T20:07:49.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.173+0000 7fdc69380700 1 -- 172.21.15.43:0/3911290019 learned_addr learned my addr 172.21.15.43:0/3911290019 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:49.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.173+0000 7fdc637fe700 1 -- 172.21.15.43:0/3911290019 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1201740614 0 0) 0x7fdc64160e90 con 0x7fdc640b1920 2023-06-28T20:07:49.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.173+0000 7fdc68b7f700 1 --2- 172.21.15.43:0/3911290019 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc640cf6e0 0x7fdc640ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:49.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.173+0000 7fdc637fe700 1 -- 172.21.15.43:0/3911290019 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc540035b0 con 0x7fdc640b1920 2023-06-28T20:07:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.173+0000 7fdc637fe700 1 -- 172.21.15.43:0/3911290019 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3827578742 0 0) 0x7fdc540035b0 con 0x7fdc640b1920 2023-06-28T20:07:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.173+0000 7fdc637fe700 1 -- 172.21.15.43:0/3911290019 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc640cf6e0 msgr2=0x7fdc640ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.173+0000 7fdc637fe700 1 --2- 172.21.15.43:0/3911290019 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc640cf6e0 0x7fdc640ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.173+0000 7fdc637fe700 1 -- 172.21.15.43:0/3911290019 >> v1:172.21.15.178:6789/0 conn(0x7fdc640cca10 legacy=0x7fdc64071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.173+0000 7fdc637fe700 1 -- 172.21.15.43:0/3911290019 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc640aed40 con 0x7fdc640b1920 2023-06-28T20:07:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.173+0000 7fdc637fe700 1 -- 172.21.15.43:0/3911290019 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7fdc5c0019f0 con 0x7fdc640b1920 2023-06-28T20:07:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.173+0000 7fdc637fe700 1 -- 172.21.15.43:0/3911290019 >> v1:172.21.15.43:6789/0 conn(0x7fdc640b1920 legacy=0x7fdc640b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.173+0000 7fdc637fe700 1 --2- 172.21.15.43:0/3911290019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc54003a30 0x7fdc54003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.173+0000 7fdc637fe700 1 --2- 172.21.15.43:0/3911290019 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc640cf6e0 0x7fdc54004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.173+0000 7fdc637fe700 1 --2- 172.21.15.43:0/3911290019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc54004d90 0x7fdc54005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.173+0000 7fdc637fe700 1 -- 172.21.15.43:0/3911290019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5400bac0 con 0x7fdc54003a30 2023-06-28T20:07:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.173+0000 7fdc637fe700 1 -- 172.21.15.43:0/3911290019 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5400bc30 con 0x7fdc54004d90 2023-06-28T20:07:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.173+0000 7fdc637fe700 1 -- 172.21.15.43:0/3911290019 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5400bdf0 con 0x7fdc640cf6e0 2023-06-28T20:07:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.173+0000 7fdc68b7f700 1 --2- 172.21.15.43:0/3911290019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc54003a30 0x7fdc54003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:49.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.173+0000 7fdc63fff700 1 --2- 172.21.15.43:0/3911290019 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc640cf6e0 0x7fdc54004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:49.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.174+0000 7fdc69380700 1 --2- 172.21.15.43:0/3911290019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc54004d90 0x7fdc54005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:49.191 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.174+0000 7fdc68b7f700 1 -- 172.21.15.43:0/3911290019 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc640cf6e0 msgr2=0x7fdc54004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.174+0000 7fdc68b7f700 1 --2- 172.21.15.43:0/3911290019 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc640cf6e0 0x7fdc54004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.174+0000 7fdc68b7f700 1 -- 172.21.15.43:0/3911290019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc54004d90 msgr2=0x7fdc54005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.174+0000 7fdc68b7f700 1 --2- 172.21.15.43:0/3911290019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc54004d90 0x7fdc54005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.174+0000 7fdc68b7f700 1 -- 172.21.15.43:0/3911290019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc540035b0 con 0x7fdc54003a30 2023-06-28T20:07:49.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.174+0000 7fdc69380700 1 --2- 172.21.15.43:0/3911290019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc54004d90 0x7fdc54005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:49.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.174+0000 7fdc68b7f700 1 --2- 172.21.15.43:0/3911290019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc54003a30 0x7fdc54003e40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009b60 tx=0x7fdc54005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fdf3f96341329d2 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:49.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.174+0000 7fdc637fe700 1 -- 172.21.15.43:0/3911290019 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5800e6c0 con 0x7fdc54003a30 2023-06-28T20:07:49.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.174+0000 7fdc637fe700 1 -- 172.21.15.43:0/3911290019 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5800e820 con 0x7fdc54003a30 2023-06-28T20:07:49.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.174+0000 7fdc637fe700 1 -- 172.21.15.43:0/3911290019 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5800eb90 con 0x7fdc54003a30 2023-06-28T20:07:49.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.174+0000 7fdc6ade3700 1 -- 172.21.15.43:0/3911290019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc54003a30 msgr2=0x7fdc54003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:49.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.174+0000 7fdc6ade3700 1 --2- 172.21.15.43:0/3911290019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc54003a30 0x7fdc54003e40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009b60 tx=0x7fdc54005880 comp rx=0 tx=0).stop 2023-06-28T20:07:49.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.175+0000 7fdc69380700 1 -- 172.21.15.43:0/3911290019 reap_dead start 2023-06-28T20:07:49.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.175+0000 7fdc6ade3700 1 -- 172.21.15.43:0/3911290019 shutdown_connections 2023-06-28T20:07:49.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.175+0000 7fdc6ade3700 1 -- 172.21.15.43:0/3911290019 >> 172.21.15.43:0/3911290019 conn(0x7fdc6406b000 msgr2=0x7fdc6406b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:49.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.175+0000 7fdc6ade3700 1 -- 172.21.15.43:0/3911290019 shutdown_connections 2023-06-28T20:07:49.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.175+0000 7fdc6ade3700 1 -- 172.21.15.43:0/3911290019 wait complete. 2023-06-28T20:07:49.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.176+0000 7fdc6ade3700 1 Processor -- start 2023-06-28T20:07:49.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.176+0000 7fdc6ade3700 1 -- start start 2023-06-28T20:07:49.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.177+0000 7fdc6ade3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc54003a30 0x7fdc6415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:49.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.177+0000 7fdc6ade3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc641e5dd0 0x7fdc641e6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.177+0000 7fdc6ade3700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc6415b860 0x7fdc6415bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.177+0000 7fdc6ade3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6415c240 con 0x7fdc641e5dd0 2023-06-28T20:07:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.177+0000 7fdc6ade3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc640c36c0 con 0x7fdc54003a30 2023-06-28T20:07:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.177+0000 7fdc6ade3700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc641e68d0 con 0x7fdc6415b860 2023-06-28T20:07:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.177+0000 7fdc63fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc641e5dd0 0x7fdc641e6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.177+0000 7fdc63fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc641e5dd0 0x7fdc641e6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53870/0 (socket says 172.21.15.43:53870) 2023-06-28T20:07:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.177+0000 7fdc63fff700 1 -- 172.21.15.43:0/3635805980 learned_addr learned my addr 172.21.15.43:0/3635805980 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.177+0000 7fdc68b7f700 1 --2- 172.21.15.43:0/3635805980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc54003a30 0x7fdc6415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.177+0000 7fdc69380700 1 --2- 172.21.15.43:0/3635805980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc6415b860 0x7fdc6415bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.177+0000 7fdc63fff700 1 -- 172.21.15.43:0/3635805980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc6415b860 msgr2=0x7fdc6415bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.177+0000 7fdc63fff700 1 --2- 172.21.15.43:0/3635805980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc6415b860 0x7fdc6415bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.177+0000 7fdc63fff700 1 -- 172.21.15.43:0/3635805980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc54003a30 msgr2=0x7fdc6415a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:49.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.177+0000 7fdc63fff700 1 --2- 172.21.15.43:0/3635805980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc54003a30 0x7fdc6415a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.177+0000 7fdc63fff700 1 -- 172.21.15.43:0/3635805980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc641e6bb0 con 0x7fdc641e5dd0 2023-06-28T20:07:49.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.177+0000 7fdc68b7f700 1 --2- 172.21.15.43:0/3635805980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc54003a30 0x7fdc6415a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:49.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.177+0000 7fdc63fff700 1 --2- 172.21.15.43:0/3635805980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc641e5dd0 0x7fdc641e6240 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000f820 tx=0x7fdc5000fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:49.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.177+0000 7fdc617fa700 1 -- 172.21.15.43:0/3635805980 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500052e0 con 0x7fdc641e5dd0 2023-06-28T20:07:49.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.177+0000 7fdc617fa700 1 -- 172.21.15.43:0/3635805980 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc50005440 con 0x7fdc641e5dd0 2023-06-28T20:07:49.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.177+0000 7fdc6ade3700 1 -- 172.21.15.43:0/3635805980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc640c3960 con 0x7fdc641e5dd0 2023-06-28T20:07:49.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.177+0000 7fdc617fa700 1 -- 172.21.15.43:0/3635805980 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500038c0 con 0x7fdc641e5dd0 2023-06-28T20:07:49.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.178+0000 7fdc6ade3700 1 -- 172.21.15.43:0/3635805980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6415b330 con 0x7fdc641e5dd0 2023-06-28T20:07:49.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.179+0000 7fdc6ade3700 1 -- 172.21.15.43:0/3635805980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc640728b0 con 0x7fdc641e5dd0 2023-06-28T20:07:49.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.179+0000 7fdc617fa700 1 -- 172.21.15.43:0/3635805980 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdc50003a20 con 0x7fdc641e5dd0 2023-06-28T20:07:49.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.179+0000 7fdc617fa700 1 --2- 172.21.15.43:0/3635805980 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fdc4806ace0 0x7fdc4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:49.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.179+0000 7fdc617fa700 1 -- 172.21.15.43:0/3635805980 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdc50089240 con 0x7fdc641e5dd0 2023-06-28T20:07:49.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.179+0000 7fdc68b7f700 1 --2- 172.21.15.43:0/3635805980 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fdc4806ace0 0x7fdc4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:49.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.180+0000 7fdc68b7f700 1 --2- 172.21.15.43:0/3635805980 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fdc4806ace0 0x7fdc4806d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdc580090b0 tx=0x7fdc58008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:49.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.184+0000 7fdc617fa700 1 -- 172.21.15.43:0/3635805980 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdc50054380 con 0x7fdc641e5dd0 2023-06-28T20:07:49.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.382+0000 7fdc6ade3700 1 -- 172.21.15.43:0/3635805980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc640cc640 con 0x7fdc641e5dd0 2023-06-28T20:07:49.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.382+0000 7fdc617fa700 1 -- 172.21.15.43:0/3635805980 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdc500579a0 con 0x7fdc641e5dd0 2023-06-28T20:07:49.386 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-06-28T20:07:49.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.384+0000 7fdc6ade3700 1 -- 172.21.15.43:0/3635805980 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fdc4806ace0 msgr2=0x7fdc4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:49.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.384+0000 7fdc6ade3700 1 --2- 172.21.15.43:0/3635805980 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fdc4806ace0 0x7fdc4806d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdc580090b0 tx=0x7fdc58008030 comp rx=0 tx=0).stop 2023-06-28T20:07:49.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.384+0000 7fdc6ade3700 1 -- 172.21.15.43:0/3635805980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc641e5dd0 msgr2=0x7fdc641e6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:49.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.384+0000 7fdc6ade3700 1 --2- 172.21.15.43:0/3635805980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc641e5dd0 0x7fdc641e6240 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000f820 tx=0x7fdc5000fbe0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.384+0000 7fdc6ade3700 1 -- 172.21.15.43:0/3635805980 shutdown_connections 2023-06-28T20:07:49.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.384+0000 7fdc6ade3700 1 --2- 172.21.15.43:0/3635805980 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fdc4806ace0 0x7fdc4806d190 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.384+0000 7fdc6ade3700 1 --2- 172.21.15.43:0/3635805980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdc6415b860 0x7fdc6415bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.384+0000 7fdc6ade3700 1 --2- 172.21.15.43:0/3635805980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc54003a30 0x7fdc6415a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.384+0000 7fdc6ade3700 1 --2- 172.21.15.43:0/3635805980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc641e5dd0 0x7fdc641e6240 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.384+0000 7fdc6ade3700 1 -- 172.21.15.43:0/3635805980 >> 172.21.15.43:0/3635805980 conn(0x7fdc6406b000 msgr2=0x7fdc6415cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:49.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.385+0000 7fdc6ade3700 1 -- 172.21.15.43:0/3635805980 shutdown_connections 2023-06-28T20:07:49.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.385+0000 7fdc6ade3700 1 -- 172.21.15.43:0/3635805980 wait complete. 2023-06-28T20:07:49.400 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:07:49.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.529+0000 7f5a22ebf700 1 Processor -- start 2023-06-28T20:07:49.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.529+0000 7f5a22ebf700 1 -- start start 2023-06-28T20:07:49.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.530+0000 7f5a22ebf700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a1c159820 0x7f5a1c15a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:49.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.530+0000 7f5a22ebf700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1c0ce0a0 con 0x7f5a1c06ee20 2023-06-28T20:07:49.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.530+0000 7f5a22ebf700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1c0ce380 con 0x7f5a1c0cc600 2023-06-28T20:07:49.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.530+0000 7f5a22ebf700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c159c90 con 0x7f5a1c159820 2023-06-28T20:07:49.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.530+0000 7f5a2145c700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5a1c06ee20 0x7f5a1c0cd7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43066/0 (socket says 172.21.15.43:43066) 2023-06-28T20:07:49.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.530+0000 7f5a2145c700 1 -- 172.21.15.43:0/2137334446 learned_addr learned my addr 172.21.15.43:0/2137334446 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.530+0000 7f5a1bfff700 1 --2- 172.21.15.43:0/2137334446 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a1c159820 0x7f5a1c15a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.530+0000 7f5a1b7fe700 1 -- 172.21.15.43:0/2137334446 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 178384237 0 0) 0x7f5a1c0ce0a0 con 0x7f5a1c06ee20 2023-06-28T20:07:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.530+0000 7f5a1b7fe700 1 -- 172.21.15.43:0/2137334446 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a0c0035b0 con 0x7f5a1c06ee20 2023-06-28T20:07:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.530+0000 7f5a1b7fe700 1 -- 172.21.15.43:0/2137334446 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 196203847 0 0) 0x7f5a1c0ce380 con 0x7f5a1c0cc600 2023-06-28T20:07:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.531+0000 7f5a1b7fe700 1 -- 172.21.15.43:0/2137334446 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a1c0ce0a0 con 0x7f5a1c0cc600 2023-06-28T20:07:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.531+0000 7f5a1bfff700 1 -- 172.21.15.43:0/2137334446 >> v1:172.21.15.178:6789/0 conn(0x7f5a1c0cc600 legacy=0x7f5a1c0cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.531+0000 7f5a1b7fe700 1 -- 172.21.15.43:0/2137334446 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1382168559 0 0) 0x7f5a0c0035b0 con 0x7f5a1c06ee20 2023-06-28T20:07:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.531+0000 7f5a1bfff700 1 -- 172.21.15.43:0/2137334446 >> v1:172.21.15.43:6789/0 conn(0x7f5a1c06ee20 legacy=0x7f5a1c0cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.531+0000 7f5a1bfff700 1 -- 172.21.15.43:0/2137334446 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c159f70 con 0x7f5a1c159820 2023-06-28T20:07:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.531+0000 7f5a1bfff700 1 --2- 172.21.15.43:0/2137334446 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a1c159820 0x7f5a1c15a490 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009870 tx=0x7f5a08009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd8dd60cdf8b5662 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.531+0000 7f5a1b7fe700 1 -- 172.21.15.43:0/2137334446 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a140019f0 con 0x7f5a1c159820 2023-06-28T20:07:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.531+0000 7f5a1b7fe700 1 -- 172.21.15.43:0/2137334446 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a08004030 con 0x7f5a1c159820 2023-06-28T20:07:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.531+0000 7f5a1b7fe700 1 -- 172.21.15.43:0/2137334446 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a080043c0 con 0x7f5a1c159820 2023-06-28T20:07:49.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.531+0000 7f5a22ebf700 1 -- 172.21.15.43:0/2137334446 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a1c159820 msgr2=0x7f5a1c15a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:49.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.531+0000 7f5a22ebf700 1 --2- 172.21.15.43:0/2137334446 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a1c159820 0x7f5a1c15a490 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009870 tx=0x7f5a08009b80 comp rx=0 tx=0).stop 2023-06-28T20:07:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.532+0000 7f5a22ebf700 1 -- 172.21.15.43:0/2137334446 shutdown_connections 2023-06-28T20:07:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.532+0000 7f5a22ebf700 1 --2- 172.21.15.43:0/2137334446 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a1c159820 0x7f5a1c15a490 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.532+0000 7f5a22ebf700 1 -- 172.21.15.43:0/2137334446 >> 172.21.15.43:0/2137334446 conn(0x7f5a1c0b1720 msgr2=0x7f5a1c0b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.532+0000 7f5a22ebf700 1 -- 172.21.15.43:0/2137334446 shutdown_connections 2023-06-28T20:07:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.532+0000 7f5a22ebf700 1 -- 172.21.15.43:0/2137334446 wait complete. 2023-06-28T20:07:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.533+0000 7f5a22ebf700 1 Processor -- start 2023-06-28T20:07:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.533+0000 7f5a22ebf700 1 -- start start 2023-06-28T20:07:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.534+0000 7f5a22ebf700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a1c06ee20 0x7f5a1c1531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.534+0000 7f5a22ebf700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a1c0cc600 0x7f5a1c0b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.534+0000 7f5a22ebf700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a1c159820 0x7f5a1c069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.534+0000 7f5a22ebf700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c0b3130 con 0x7f5a1c159820 2023-06-28T20:07:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.534+0000 7f5a22ebf700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c153720 con 0x7f5a1c0cc600 2023-06-28T20:07:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.534+0000 7f5a22ebf700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c155750 con 0x7f5a1c06ee20 2023-06-28T20:07:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.534+0000 7f5a2145c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a1c159820 0x7f5a1c069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.534+0000 7f5a2145c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a1c159820 0x7f5a1c069ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53886/0 (socket says 172.21.15.43:53886) 2023-06-28T20:07:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.534+0000 7f5a2145c700 1 -- 172.21.15.43:0/423402615 learned_addr learned my addr 172.21.15.43:0/423402615 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.534+0000 7f5a20c5b700 1 --2- 172.21.15.43:0/423402615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a1c06ee20 0x7f5a1c1531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.534+0000 7f5a1bfff700 1 --2- 172.21.15.43:0/423402615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a1c0cc600 0x7f5a1c0b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.534+0000 7f5a2145c700 1 -- 172.21.15.43:0/423402615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a1c06ee20 msgr2=0x7f5a1c1531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.534+0000 7f5a2145c700 1 --2- 172.21.15.43:0/423402615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a1c06ee20 0x7f5a1c1531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.534+0000 7f5a2145c700 1 -- 172.21.15.43:0/423402615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a1c0cc600 msgr2=0x7f5a1c0b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.534+0000 7f5a2145c700 1 --2- 172.21.15.43:0/423402615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a1c0cc600 0x7f5a1c0b35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.534+0000 7f5a2145c700 1 -- 172.21.15.43:0/423402615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a080094a0 con 0x7f5a1c159820 2023-06-28T20:07:49.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.534+0000 7f5a2145c700 1 --2- 172.21.15.43:0/423402615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a1c159820 0x7f5a1c069ef0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5a1400cc80 tx=0x7f5a1400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:49.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.534+0000 7f5a197fa700 1 -- 172.21.15.43:0/423402615 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1400ada0 con 0x7f5a1c159820 2023-06-28T20:07:49.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.535+0000 7f5a197fa700 1 -- 172.21.15.43:0/423402615 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1400ce80 con 0x7f5a1c159820 2023-06-28T20:07:49.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.535+0000 7f5a22ebf700 1 -- 172.21.15.43:0/423402615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1c155a30 con 0x7f5a1c159820 2023-06-28T20:07:49.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.535+0000 7f5a197fa700 1 -- 172.21.15.43:0/423402615 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a14007310 con 0x7f5a1c159820 2023-06-28T20:07:49.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.535+0000 7f5a22ebf700 1 -- 172.21.15.43:0/423402615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1c0b2dc0 con 0x7f5a1c159820 2023-06-28T20:07:49.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.536+0000 7f5a22ebf700 1 -- 172.21.15.43:0/423402615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a1c0728b0 con 0x7f5a1c159820 2023-06-28T20:07:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.536+0000 7f5a197fa700 1 -- 172.21.15.43:0/423402615 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5a14007470 con 0x7f5a1c159820 2023-06-28T20:07:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.537+0000 7f5a197fa700 1 --2- 172.21.15.43:0/423402615 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5a0006ace0 0x7f5a0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.537+0000 7f5a20c5b700 1 --2- 172.21.15.43:0/423402615 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5a0006ace0 0x7f5a0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.537+0000 7f5a197fa700 1 -- 172.21.15.43:0/423402615 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5a140894c0 con 0x7f5a1c159820 2023-06-28T20:07:49.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.537+0000 7f5a20c5b700 1 --2- 172.21.15.43:0/423402615 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5a0006ace0 0x7f5a0006d190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5a10001220 tx=0x7f5a10009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:49.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.541+0000 7f5a197fa700 1 -- 172.21.15.43:0/423402615 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5a14054570 con 0x7f5a1c159820 2023-06-28T20:07:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.740+0000 7f5a22ebf700 1 -- 172.21.15.43:0/423402615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a1c0b3270 con 0x7f5a1c159820 2023-06-28T20:07:49.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.741+0000 7f5a197fa700 1 -- 172.21.15.43:0/423402615 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18221 (secure 0 0 0) 0x7f5a14057b90 con 0x7f5a1c159820 2023-06-28T20:07:49.745 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:49.745 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:07:22.377248+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:07:18.624759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-28T20:07:19.760017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:07:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.743+0000 7f5a22ebf700 1 -- 172.21.15.43:0/423402615 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5a0006ace0 msgr2=0x7f5a0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.744+0000 7f5a22ebf700 1 --2- 172.21.15.43:0/423402615 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5a0006ace0 0x7f5a0006d190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5a10001220 tx=0x7f5a10009040 comp rx=0 tx=0).stop 2023-06-28T20:07:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.744+0000 7f5a22ebf700 1 -- 172.21.15.43:0/423402615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a1c159820 msgr2=0x7f5a1c069ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.744+0000 7f5a22ebf700 1 --2- 172.21.15.43:0/423402615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a1c159820 0x7f5a1c069ef0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5a1400cc80 tx=0x7f5a1400a340 comp rx=0 tx=0).stop 2023-06-28T20:07:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.744+0000 7f5a22ebf700 1 -- 172.21.15.43:0/423402615 shutdown_connections 2023-06-28T20:07:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.744+0000 7f5a22ebf700 1 --2- 172.21.15.43:0/423402615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a1c159820 0x7f5a1c069ef0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.744+0000 7f5a22ebf700 1 --2- 172.21.15.43:0/423402615 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5a0006ace0 0x7f5a0006d190 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.744+0000 7f5a22ebf700 1 --2- 172.21.15.43:0/423402615 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5a1c06ee20 0x7f5a1c1531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.744+0000 7f5a22ebf700 1 --2- 172.21.15.43:0/423402615 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a1c0cc600 0x7f5a1c0b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.744+0000 7f5a22ebf700 1 -- 172.21.15.43:0/423402615 >> 172.21.15.43:0/423402615 conn(0x7f5a1c0b1720 msgr2=0x7f5a1c0cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:49.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.744+0000 7f5a22ebf700 1 -- 172.21.15.43:0/423402615 shutdown_connections 2023-06-28T20:07:49.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.744+0000 7f5a22ebf700 1 -- 172.21.15.43:0/423402615 wait complete. 2023-06-28T20:07:49.760 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-28T20:07:13.580969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-28T20:07:14.038358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-28T20:07:18.624759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-28T20:07:19.760017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-28T20:07:49.761 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-28T20:07:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.885+0000 7fd3a6268700 1 Processor -- start 2023-06-28T20:07:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.885+0000 7fd3a6268700 1 -- start start 2023-06-28T20:07:49.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.886+0000 7fd3a6268700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd3a01592d0 0x7fd3a00b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:49.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.886+0000 7fd3a6268700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3a01596a0 0x7fd3a00b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:49.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.886+0000 7fd3a6268700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd3a0071900 0x7fd3a015d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:49.890 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.886+0000 7fd3a6268700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3a0070b40 con 0x7fd3a0071900 2023-06-28T20:07:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.886+0000 7fd3a6268700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3a0070e20 con 0x7fd3a01592d0 2023-06-28T20:07:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.886+0000 7fd3a6268700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a0071100 con 0x7fd3a01596a0 2023-06-28T20:07:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.886+0000 7fd3a4805700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd3a0071900 0x7fd3a015d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.886+0000 7fd3a4805700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd3a0071900 0x7fd3a015d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53894/0 (socket says 172.21.15.43:53894) 2023-06-28T20:07:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.886+0000 7fd3a4805700 1 -- 172.21.15.43:0/1058177715 learned_addr learned my addr 172.21.15.43:0/1058177715 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.886+0000 7fd39f7fe700 1 --2- 172.21.15.43:0/1058177715 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3a01596a0 0x7fd3a00b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.887+0000 7fd39ffff700 1 --2- 172.21.15.43:0/1058177715 >> v2:172.21.15.178:3300/0 conn(0x7fd3a01592d0 0x7fd3a00b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.887+0000 7fd3a4805700 1 -- 172.21.15.43:0/1058177715 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3a01596a0 msgr2=0x7fd3a00b1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.887+0000 7fd3a4805700 1 --2- 172.21.15.43:0/1058177715 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3a01596a0 0x7fd3a00b1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.887+0000 7fd3a4805700 1 -- 172.21.15.43:0/1058177715 >> v2:172.21.15.178:3300/0 conn(0x7fd3a01592d0 msgr2=0x7fd3a00b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.887+0000 7fd3a4805700 1 --2- 172.21.15.43:0/1058177715 >> v2:172.21.15.178:3300/0 conn(0x7fd3a01592d0 0x7fd3a00b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.887+0000 7fd3a4805700 1 -- 172.21.15.43:0/1058177715 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a00713e0 con 0x7fd3a0071900 2023-06-28T20:07:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.887+0000 7fd3a4805700 1 --2- 172.21.15.43:0/1058177715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a0071900 0x7fd3a015d070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd3a00d0bf0 tx=0x7fd39800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd540f9ebe88ab8 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.887+0000 7fd39effd700 1 -- 172.21.15.43:0/1058177715 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39800e070 con 0x7fd3a0071900 2023-06-28T20:07:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.887+0000 7fd39effd700 1 -- 172.21.15.43:0/1058177715 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39800b910 con 0x7fd3a0071900 2023-06-28T20:07:49.893 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.887+0000 7fd39effd700 1 -- 172.21.15.43:0/1058177715 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39800bcc0 con 0x7fd3a0071900 2023-06-28T20:07:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.888+0000 7fd3a6268700 1 -- 172.21.15.43:0/1058177715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a0071900 msgr2=0x7fd3a015d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.888+0000 7fd3a6268700 1 --2- 172.21.15.43:0/1058177715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a0071900 0x7fd3a015d070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd3a00d0bf0 tx=0x7fd39800b330 comp rx=0 tx=0).stop 2023-06-28T20:07:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.888+0000 7fd3a6268700 1 -- 172.21.15.43:0/1058177715 shutdown_connections 2023-06-28T20:07:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.888+0000 7fd3a6268700 1 --2- 172.21.15.43:0/1058177715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a0071900 0x7fd3a015d070 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.888+0000 7fd3a6268700 1 --2- 172.21.15.43:0/1058177715 >> v2:172.21.15.178:3300/0 conn(0x7fd3a01592d0 0x7fd3a00b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.888+0000 7fd3a6268700 1 --2- 172.21.15.43:0/1058177715 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3a01596a0 0x7fd3a00b1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.888+0000 7fd3a6268700 1 -- 172.21.15.43:0/1058177715 >> 172.21.15.43:0/1058177715 conn(0x7fd3a006b000 msgr2=0x7fd3a006b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:49.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.888+0000 7fd3a6268700 1 -- 172.21.15.43:0/1058177715 shutdown_connections 2023-06-28T20:07:49.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.888+0000 7fd3a6268700 1 -- 172.21.15.43:0/1058177715 wait complete. 2023-06-28T20:07:49.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.889+0000 7fd3a6268700 1 Processor -- start 2023-06-28T20:07:49.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.889+0000 7fd3a6268700 1 -- start start 2023-06-28T20:07:49.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.890+0000 7fd3a6268700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a0071900 0x7fd3a00cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:49.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.890+0000 7fd3a6268700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3a01592d0 0x7fd3a0149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:49.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.890+0000 7fd3a6268700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3a01596a0 0x7fd3a00cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:49.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.890+0000 7fd3a6268700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a015a440 con 0x7fd3a0071900 2023-06-28T20:07:49.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.890+0000 7fd3a6268700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a0166200 con 0x7fd3a01592d0 2023-06-28T20:07:49.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.890+0000 7fd3a6268700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a0159aa0 con 0x7fd3a01596a0 2023-06-28T20:07:49.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.890+0000 7fd39ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a0071900 0x7fd3a00cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:49.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.890+0000 7fd39ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a0071900 0x7fd3a00cb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53896/0 (socket says 172.21.15.43:53896) 2023-06-28T20:07:49.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.890+0000 7fd39ffff700 1 -- 172.21.15.43:0/986360410 learned_addr learned my addr 172.21.15.43:0/986360410 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:49.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.890+0000 7fd3a4805700 1 --2- 172.21.15.43:0/986360410 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3a01596a0 0x7fd3a00cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:49.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.890+0000 7fd39f7fe700 1 --2- 172.21.15.43:0/986360410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3a01592d0 0x7fd3a0149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:49.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.890+0000 7fd39ffff700 1 -- 172.21.15.43:0/986360410 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3a01596a0 msgr2=0x7fd3a00cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:49.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.890+0000 7fd39ffff700 1 --2- 172.21.15.43:0/986360410 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3a01596a0 0x7fd3a00cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.890+0000 7fd39ffff700 1 -- 172.21.15.43:0/986360410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3a01592d0 msgr2=0x7fd3a0149040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:49.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.890+0000 7fd39ffff700 1 --2- 172.21.15.43:0/986360410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3a01592d0 0x7fd3a0149040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:49.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.890+0000 7fd39ffff700 1 -- 172.21.15.43:0/986360410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd398009b30 con 0x7fd3a0071900 2023-06-28T20:07:49.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.890+0000 7fd39f7fe700 1 --2- 172.21.15.43:0/986360410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3a01592d0 0x7fd3a0149040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:49.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.890+0000 7fd39ffff700 1 --2- 172.21.15.43:0/986360410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a0071900 0x7fd3a00cb750 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd39000e930 tx=0x7fd39000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:49.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.891+0000 7fd39cff9700 1 -- 172.21.15.43:0/986360410 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39000ee80 con 0x7fd3a0071900 2023-06-28T20:07:49.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.891+0000 7fd39cff9700 1 -- 172.21.15.43:0/986360410 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39000c850 con 0x7fd3a0071900 2023-06-28T20:07:49.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.891+0000 7fd3a6268700 1 -- 172.21.15.43:0/986360410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a0159d80 con 0x7fd3a0071900 2023-06-28T20:07:49.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.891+0000 7fd39cff9700 1 -- 172.21.15.43:0/986360410 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39000cbe0 con 0x7fd3a0071900 2023-06-28T20:07:49.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.891+0000 7fd3a6268700 1 -- 172.21.15.43:0/986360410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a0165e90 con 0x7fd3a0071900 2023-06-28T20:07:49.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.892+0000 7fd3a6268700 1 -- 172.21.15.43:0/986360410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a00c69a0 con 0x7fd3a0071900 2023-06-28T20:07:49.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.892+0000 7fd39cff9700 1 -- 172.21.15.43:0/986360410 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd39000cd40 con 0x7fd3a0071900 2023-06-28T20:07:49.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.892+0000 7fd39cff9700 1 --2- 172.21.15.43:0/986360410 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd38006ace0 0x7fd38006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:49.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.893+0000 7fd39f7fe700 1 --2- 172.21.15.43:0/986360410 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd38006ace0 0x7fd38006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.893+0000 7fd39cff9700 1 -- 172.21.15.43:0/986360410 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd390089500 con 0x7fd3a0071900 2023-06-28T20:07:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.893+0000 7fd39f7fe700 1 --2- 172.21.15.43:0/986360410 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd38006ace0 0x7fd38006d190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd3a014f7a0 tx=0x7fd388008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:49.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:49.897+0000 7fd39cff9700 1 -- 172.21.15.43:0/986360410 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd3900545b0 con 0x7fd3a0071900 2023-06-28T20:07:50.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.094+0000 7fd3a6268700 1 -- 172.21.15.43:0/986360410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd3a00cc3e0 con 0x7fd3a0071900 2023-06-28T20:07:50.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.094+0000 7fd39cff9700 1 -- 172.21.15.43:0/986360410 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd390057bd0 con 0x7fd3a0071900 2023-06-28T20:07:50.097 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-06-28T20:07:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.096+0000 7fd3a6268700 1 -- 172.21.15.43:0/986360410 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd38006ace0 msgr2=0x7fd38006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.096+0000 7fd3a6268700 1 --2- 172.21.15.43:0/986360410 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd38006ace0 0x7fd38006d190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd3a014f7a0 tx=0x7fd388008040 comp rx=0 tx=0).stop 2023-06-28T20:07:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.096+0000 7fd3a6268700 1 -- 172.21.15.43:0/986360410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a0071900 msgr2=0x7fd3a00cb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.096+0000 7fd3a6268700 1 --2- 172.21.15.43:0/986360410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a0071900 0x7fd3a00cb750 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd39000e930 tx=0x7fd39000ec40 comp rx=0 tx=0).stop 2023-06-28T20:07:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.096+0000 7fd3a6268700 1 -- 172.21.15.43:0/986360410 shutdown_connections 2023-06-28T20:07:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.096+0000 7fd3a6268700 1 --2- 172.21.15.43:0/986360410 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd38006ace0 0x7fd38006d190 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.096+0000 7fd3a6268700 1 --2- 172.21.15.43:0/986360410 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd3a01596a0 0x7fd3a00cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.096+0000 7fd3a6268700 1 --2- 172.21.15.43:0/986360410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3a0071900 0x7fd3a00cb750 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.096+0000 7fd3a6268700 1 --2- 172.21.15.43:0/986360410 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd3a01592d0 0x7fd3a0149040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.096+0000 7fd3a6268700 1 -- 172.21.15.43:0/986360410 >> 172.21.15.43:0/986360410 conn(0x7fd3a006b000 msgr2=0x7fd3a014fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:50.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.096+0000 7fd3a6268700 1 -- 172.21.15.43:0/986360410 shutdown_connections 2023-06-28T20:07:50.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.096+0000 7fd3a6268700 1 -- 172.21.15.43:0/986360410 wait complete. 2023-06-28T20:07:50.111 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-28T20:07:50.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.236+0000 7f0a15f95700 1 Processor -- start 2023-06-28T20:07:50.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.237+0000 7f0a15f95700 1 -- start start 2023-06-28T20:07:50.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.237+0000 7f0a15f95700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a100af060 0x7f0a10160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:50.242 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.237+0000 7f0a15f95700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0a1006ccc0 0x7f0a100b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:50.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.237+0000 7f0a15f95700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a10160250 con 0x7f0a100aec90 2023-06-28T20:07:50.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.237+0000 7f0a15f95700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a1015cb30 con 0x7f0a1006ccc0 2023-06-28T20:07:50.243 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.237+0000 7f0a15f95700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a10071410 con 0x7f0a100af060 2023-06-28T20:07:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.238+0000 7f0a0effd700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a100af060 0x7f0a10160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.238+0000 7f0a0f7fe700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f0a100aec90 0x7f0a10070bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:42212/0 (socket says 172.21.15.43:42212) 2023-06-28T20:07:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.238+0000 7f0a0effd700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a100af060 0x7f0a10160cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.43:37876/0 (socket says 172.21.15.43:37876) 2023-06-28T20:07:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.238+0000 7f0a0ffff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0a1006ccc0 0x7f0a100b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:50.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.238+0000 7f0a0f7fe700 1 -- 172.21.15.43:0/4205600656 learned_addr learned my addr 172.21.15.43:0/4205600656 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:50.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.238+0000 7f0a0ffff700 1 -- 172.21.15.43:0/4205600656 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a100af060 msgr2=0x7f0a10160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:50.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.238+0000 7f0a0ffff700 1 --2- 172.21.15.43:0/4205600656 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a100af060 0x7f0a10160cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.238+0000 7f0a0ffff700 1 -- 172.21.15.43:0/4205600656 >> v1:172.21.15.178:6789/0 conn(0x7f0a100aec90 legacy=0x7f0a10070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:50.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.238+0000 7f0a0ffff700 1 -- 172.21.15.43:0/4205600656 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a101604f0 con 0x7f0a1006ccc0 2023-06-28T20:07:50.246 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.238+0000 7f0a0effd700 1 --2- 172.21.15.43:0/4205600656 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a100af060 0x7f0a10160cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:50.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.238+0000 7f0a0ffff700 1 --2- 172.21.15.43:0/4205600656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a1006ccc0 0x7f0a100b1880 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0a08009d90 tx=0x7f0a0800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20af09ef04aed0e9 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:50.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.238+0000 7f0a0e7fc700 1 -- 172.21.15.43:0/4205600656 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0800e070 con 0x7f0a1006ccc0 2023-06-28T20:07:50.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.239+0000 7f0a0e7fc700 1 -- 172.21.15.43:0/4205600656 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0800bb20 con 0x7f0a1006ccc0 2023-06-28T20:07:50.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.239+0000 7f0a0e7fc700 1 -- 172.21.15.43:0/4205600656 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a08012430 con 0x7f0a1006ccc0 2023-06-28T20:07:50.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.239+0000 7f0a15f95700 1 -- 172.21.15.43:0/4205600656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a1006ccc0 msgr2=0x7f0a100b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:50.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.239+0000 7f0a15f95700 1 --2- 172.21.15.43:0/4205600656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a1006ccc0 0x7f0a100b1880 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0a08009d90 tx=0x7f0a0800b330 comp rx=0 tx=0).stop 2023-06-28T20:07:50.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.239+0000 7f0a15f95700 1 -- 172.21.15.43:0/4205600656 shutdown_connections 2023-06-28T20:07:50.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.239+0000 7f0a15f95700 1 --2- 172.21.15.43:0/4205600656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a1006ccc0 0x7f0a100b1880 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.239+0000 7f0a15f95700 1 --2- 172.21.15.43:0/4205600656 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a100af060 0x7f0a10160cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.239+0000 7f0a15f95700 1 -- 172.21.15.43:0/4205600656 >> 172.21.15.43:0/4205600656 conn(0x7f0a1006b000 msgr2=0x7f0a1006b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:50.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.239+0000 7f0a15f95700 1 -- 172.21.15.43:0/4205600656 shutdown_connections 2023-06-28T20:07:50.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.239+0000 7f0a15f95700 1 -- 172.21.15.43:0/4205600656 wait complete. 2023-06-28T20:07:50.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.240+0000 7f0a15f95700 1 Processor -- start 2023-06-28T20:07:50.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.241+0000 7f0a15f95700 1 -- start start 2023-06-28T20:07:50.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.241+0000 7f0a15f95700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a1006ccc0 0x7f0a100b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:50.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.241+0000 7f0a15f95700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a100aec90 0x7f0a100b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:50.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.241+0000 7f0a15f95700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a100af060 0x7f0a1014ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:50.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.241+0000 7f0a15f95700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a100b3bc0 con 0x7f0a1006ccc0 2023-06-28T20:07:50.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.241+0000 7f0a15f95700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a100b2fb0 con 0x7f0a100aec90 2023-06-28T20:07:50.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.241+0000 7f0a15f95700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a100b3290 con 0x7f0a100af060 2023-06-28T20:07:50.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.241+0000 7f0a0f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a1006ccc0 0x7f0a100b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:50.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.241+0000 7f0a0f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a1006ccc0 0x7f0a100b2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53908/0 (socket says 172.21.15.43:53908) 2023-06-28T20:07:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.241+0000 7f0a0f7fe700 1 -- 172.21.15.43:0/3507397500 learned_addr learned my addr 172.21.15.43:0/3507397500 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.242+0000 7f0a0effd700 1 --2- 172.21.15.43:0/3507397500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a100aec90 0x7f0a100b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.242+0000 7f0a0ffff700 1 --2- 172.21.15.43:0/3507397500 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a100af060 0x7f0a1014ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.242+0000 7f0a0f7fe700 1 -- 172.21.15.43:0/3507397500 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a100af060 msgr2=0x7f0a1014ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.242+0000 7f0a0f7fe700 1 --2- 172.21.15.43:0/3507397500 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a100af060 0x7f0a1014ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.242+0000 7f0a0f7fe700 1 -- 172.21.15.43:0/3507397500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a100aec90 msgr2=0x7f0a100b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.242+0000 7f0a0f7fe700 1 --2- 172.21.15.43:0/3507397500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a100aec90 0x7f0a100b4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.242+0000 7f0a0f7fe700 1 -- 172.21.15.43:0/3507397500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a000094a0 con 0x7f0a1006ccc0 2023-06-28T20:07:50.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.242+0000 7f0a0ffff700 1 --2- 172.21.15.43:0/3507397500 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a100af060 0x7f0a1014ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:50.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.242+0000 7f0a0f7fe700 1 --2- 172.21.15.43:0/3507397500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a1006ccc0 0x7f0a100b2940 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f09f800c520 tx=0x7f09f800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:50.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.242+0000 7f0a07fff700 1 -- 172.21.15.43:0/3507397500 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8005d70 con 0x7f0a1006ccc0 2023-06-28T20:07:50.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.242+0000 7f0a07fff700 1 -- 172.21.15.43:0/3507397500 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f80070d0 con 0x7f0a1006ccc0 2023-06-28T20:07:50.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.242+0000 7f0a15f95700 1 -- 172.21.15.43:0/3507397500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a08009a70 con 0x7f0a1006ccc0 2023-06-28T20:07:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.242+0000 7f0a07fff700 1 -- 172.21.15.43:0/3507397500 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8007440 con 0x7f0a1006ccc0 2023-06-28T20:07:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.242+0000 7f0a15f95700 1 -- 172.21.15.43:0/3507397500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a100b3850 con 0x7f0a1006ccc0 2023-06-28T20:07:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.243+0000 7f0a15f95700 1 -- 172.21.15.43:0/3507397500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a100b3ea0 con 0x7f0a1006ccc0 2023-06-28T20:07:50.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.243+0000 7f0a07fff700 1 -- 172.21.15.43:0/3507397500 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f09f80075a0 con 0x7f0a1006ccc0 2023-06-28T20:07:50.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.244+0000 7f0a07fff700 1 --2- 172.21.15.43:0/3507397500 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f09f006ace0 0x7f09f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:50.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.244+0000 7f0a07fff700 1 -- 172.21.15.43:0/3507397500 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f09f8088800 con 0x7f0a1006ccc0 2023-06-28T20:07:50.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.244+0000 7f0a0effd700 1 --2- 172.21.15.43:0/3507397500 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f09f006ace0 0x7f09f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.244+0000 7f0a0effd700 1 --2- 172.21.15.43:0/3507397500 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f09f006ace0 0x7f09f006d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0a101f5e70 tx=0x7f0a00008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.248+0000 7f0a07fff700 1 -- 172.21.15.43:0/3507397500 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f09f80539f0 con 0x7f0a1006ccc0 2023-06-28T20:07:50.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.447+0000 7f0a15f95700 1 -- 172.21.15.43:0/3507397500 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a1015c600 con 0x7f0a1006ccc0 2023-06-28T20:07:50.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.448+0000 7f0a07fff700 1 -- 172.21.15.43:0/3507397500 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09f8057010 con 0x7f0a1006ccc0 2023-06-28T20:07:50.452 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-06-28T20:07:50.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.450+0000 7f0a15f95700 1 -- 172.21.15.43:0/3507397500 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f09f006ace0 msgr2=0x7f09f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:50.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.450+0000 7f0a15f95700 1 --2- 172.21.15.43:0/3507397500 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f09f006ace0 0x7f09f006d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0a101f5e70 tx=0x7f0a00008040 comp rx=0 tx=0).stop 2023-06-28T20:07:50.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.450+0000 7f0a15f95700 1 -- 172.21.15.43:0/3507397500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a1006ccc0 msgr2=0x7f0a100b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:50.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.450+0000 7f0a15f95700 1 --2- 172.21.15.43:0/3507397500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a1006ccc0 0x7f0a100b2940 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f09f800c520 tx=0x7f09f800c8e0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.450+0000 7f0a15f95700 1 -- 172.21.15.43:0/3507397500 shutdown_connections 2023-06-28T20:07:50.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.450+0000 7f0a15f95700 1 --2- 172.21.15.43:0/3507397500 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f09f006ace0 0x7f09f006d190 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.450+0000 7f0a15f95700 1 --2- 172.21.15.43:0/3507397500 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0a100af060 0x7f0a1014ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.450+0000 7f0a15f95700 1 --2- 172.21.15.43:0/3507397500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a1006ccc0 0x7f0a100b2940 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.450+0000 7f0a15f95700 1 --2- 172.21.15.43:0/3507397500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a100aec90 0x7f0a100b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.450+0000 7f0a15f95700 1 -- 172.21.15.43:0/3507397500 >> 172.21.15.43:0/3507397500 conn(0x7f0a1006b000 msgr2=0x7f0a100adec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:50.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.450+0000 7f0a15f95700 1 -- 172.21.15.43:0/3507397500 shutdown_connections 2023-06-28T20:07:50.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.450+0000 7f0a15f95700 1 -- 172.21.15.43:0/3507397500 wait complete. 2023-06-28T20:07:50.466 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-28T20:07:50.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.592+0000 7f609f196700 1 Processor -- start 2023-06-28T20:07:50.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.592+0000 7f609f196700 1 -- start start 2023-06-28T20:07:50.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.593+0000 7f609f196700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60980aec90 0x7f6098160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:50.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.593+0000 7f609f196700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6098160250 con 0x7f60980af060 2023-06-28T20:07:50.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.593+0000 7f609f196700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6098071410 con 0x7f609806ccc0 2023-06-28T20:07:50.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.593+0000 7f609f196700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609815cb30 con 0x7f60980aec90 2023-06-28T20:07:50.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.593+0000 7f6097fff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f60980af060 0x7f6098070bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43102/0 (socket says 172.21.15.43:43102) 2023-06-28T20:07:50.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.593+0000 7f6097fff700 1 -- 172.21.15.43:0/2899456217 learned_addr learned my addr 172.21.15.43:0/2899456217 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:50.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.593+0000 7f609cf32700 1 --2- 172.21.15.43:0/2899456217 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60980aec90 0x7f6098160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:50.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.594+0000 7f60977fe700 1 -- 172.21.15.43:0/2899456217 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 111283242 0 0) 0x7f6098160250 con 0x7f60980af060 2023-06-28T20:07:50.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.594+0000 7f60977fe700 1 -- 172.21.15.43:0/2899456217 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60880035b0 con 0x7f60980af060 2023-06-28T20:07:50.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.594+0000 7f60977fe700 1 -- 172.21.15.43:0/2899456217 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3960278732 0 0) 0x7f6098071410 con 0x7f609806ccc0 2023-06-28T20:07:50.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.594+0000 7f60977fe700 1 -- 172.21.15.43:0/2899456217 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6098160250 con 0x7f609806ccc0 2023-06-28T20:07:50.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.594+0000 7f609cf32700 1 -- 172.21.15.43:0/2899456217 >> v1:172.21.15.178:6789/0 conn(0x7f609806ccc0 legacy=0x7f60980b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:50.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.594+0000 7f609cf32700 1 -- 172.21.15.43:0/2899456217 >> v1:172.21.15.43:6789/0 conn(0x7f60980af060 legacy=0x7f6098070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:50.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.594+0000 7f609cf32700 1 -- 172.21.15.43:0/2899456217 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60981604f0 con 0x7f60980aec90 2023-06-28T20:07:50.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.594+0000 7f609cf32700 1 --2- 172.21.15.43:0/2899456217 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60980aec90 0x7f6098160cf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f608c009870 tx=0x7f608c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=798e14ee57c9dd5f server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:50.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.594+0000 7f60977fe700 1 -- 172.21.15.43:0/2899456217 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c011830 con 0x7f60980aec90 2023-06-28T20:07:50.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.594+0000 7f60977fe700 1 -- 172.21.15.43:0/2899456217 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608c011990 con 0x7f60980aec90 2023-06-28T20:07:50.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.594+0000 7f60977fe700 1 -- 172.21.15.43:0/2899456217 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c011d20 con 0x7f60980aec90 2023-06-28T20:07:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.595+0000 7f609f196700 1 -- 172.21.15.43:0/2899456217 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60980aec90 msgr2=0x7f6098160cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.595+0000 7f609f196700 1 --2- 172.21.15.43:0/2899456217 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60980aec90 0x7f6098160cf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f608c009870 tx=0x7f608c009b80 comp rx=0 tx=0).stop 2023-06-28T20:07:50.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.595+0000 7f609f196700 1 -- 172.21.15.43:0/2899456217 shutdown_connections 2023-06-28T20:07:50.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.595+0000 7f609f196700 1 --2- 172.21.15.43:0/2899456217 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60980aec90 0x7f6098160cf0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.595+0000 7f609f196700 1 -- 172.21.15.43:0/2899456217 >> 172.21.15.43:0/2899456217 conn(0x7f609806b000 msgr2=0x7f609806b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:50.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.595+0000 7f609f196700 1 -- 172.21.15.43:0/2899456217 shutdown_connections 2023-06-28T20:07:50.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.595+0000 7f609f196700 1 -- 172.21.15.43:0/2899456217 wait complete. 2023-06-28T20:07:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.596+0000 7f609f196700 1 Processor -- start 2023-06-28T20:07:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.596+0000 7f609f196700 1 -- start start 2023-06-28T20:07:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.597+0000 7f609f196700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f609806ccc0 0x7f609815dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.597+0000 7f609f196700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60980aec90 0x7f609815ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.597+0000 7f609f196700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60980af060 0x7f60980c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.597+0000 7f609f196700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609815e510 con 0x7f60980af060 2023-06-28T20:07:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.597+0000 7f609f196700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60981f5a30 con 0x7f609806ccc0 2023-06-28T20:07:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.597+0000 7f609f196700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60981f5d10 con 0x7f60980aec90 2023-06-28T20:07:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.597+0000 7f609d733700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60980af060 0x7f60980c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.597+0000 7f609d733700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60980af060 0x7f60980c5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53926/0 (socket says 172.21.15.43:53926) 2023-06-28T20:07:50.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.597+0000 7f609d733700 1 -- 172.21.15.43:0/4181133539 learned_addr learned my addr 172.21.15.43:0/4181133539 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:50.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.597+0000 7f609cf32700 1 --2- 172.21.15.43:0/4181133539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f609806ccc0 0x7f609815dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:50.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.597+0000 7f6097fff700 1 --2- 172.21.15.43:0/4181133539 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60980aec90 0x7f609815ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:50.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.597+0000 7f609d733700 1 -- 172.21.15.43:0/4181133539 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60980aec90 msgr2=0x7f609815ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.597+0000 7f609d733700 1 --2- 172.21.15.43:0/4181133539 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60980aec90 0x7f609815ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.597+0000 7f609d733700 1 -- 172.21.15.43:0/4181133539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f609806ccc0 msgr2=0x7f609815dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.597+0000 7f609d733700 1 --2- 172.21.15.43:0/4181133539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f609806ccc0 0x7f609815dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.597+0000 7f609d733700 1 -- 172.21.15.43:0/4181133539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c0094a0 con 0x7f60980af060 2023-06-28T20:07:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.598+0000 7f609d733700 1 --2- 172.21.15.43:0/4181133539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60980af060 0x7f60980c5e30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f609000cc20 tx=0x7f609000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.598+0000 7f60957fa700 1 -- 172.21.15.43:0/4181133539 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609000ac20 con 0x7f60980af060 2023-06-28T20:07:50.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.598+0000 7f60957fa700 1 -- 172.21.15.43:0/4181133539 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609000ad80 con 0x7f60980af060 2023-06-28T20:07:50.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.598+0000 7f609f196700 1 -- 172.21.15.43:0/4181133539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60981f5ff0 con 0x7f60980af060 2023-06-28T20:07:50.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.598+0000 7f60957fa700 1 -- 172.21.15.43:0/4181133539 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090003c20 con 0x7f60980af060 2023-06-28T20:07:50.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.598+0000 7f609f196700 1 -- 172.21.15.43:0/4181133539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60981f56b0 con 0x7f60980af060 2023-06-28T20:07:50.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.599+0000 7f609f196700 1 -- 172.21.15.43:0/4181133539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60980728b0 con 0x7f60980af060 2023-06-28T20:07:50.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.599+0000 7f60957fa700 1 -- 172.21.15.43:0/4181133539 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f60900050d0 con 0x7f60980af060 2023-06-28T20:07:50.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.599+0000 7f60957fa700 1 --2- 172.21.15.43:0/4181133539 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f607c06ad10 0x7f607c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:50.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.600+0000 7f60957fa700 1 -- 172.21.15.43:0/4181133539 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f60900896b0 con 0x7f60980af060 2023-06-28T20:07:50.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.600+0000 7f609cf32700 1 --2- 172.21.15.43:0/4181133539 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f607c06ad10 0x7f607c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:50.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.600+0000 7f609cf32700 1 --2- 172.21.15.43:0/4181133539 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f607c06ad10 0x7f607c06d1c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f608c009bc0 tx=0x7f608c01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:50.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.604+0000 7f60957fa700 1 -- 172.21.15.43:0/4181133539 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f60900546c0 con 0x7f60980af060 2023-06-28T20:07:50.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.801+0000 7f609f196700 1 -- 172.21.15.43:0/4181133539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f609806d9c0 con 0x7f60980af060 2023-06-28T20:07:50.805 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.802+0000 7f60957fa700 1 -- 172.21.15.43:0/4181133539 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6090057ce0 con 0x7f60980af060 2023-06-28T20:07:50.805 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-06-28T20:07:50.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.804+0000 7f609f196700 1 -- 172.21.15.43:0/4181133539 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f607c06ad10 msgr2=0x7f607c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:50.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.804+0000 7f609f196700 1 --2- 172.21.15.43:0/4181133539 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f607c06ad10 0x7f607c06d1c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f608c009bc0 tx=0x7f608c01c040 comp rx=0 tx=0).stop 2023-06-28T20:07:50.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.804+0000 7f609f196700 1 -- 172.21.15.43:0/4181133539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60980af060 msgr2=0x7f60980c5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:50.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.804+0000 7f609f196700 1 --2- 172.21.15.43:0/4181133539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60980af060 0x7f60980c5e30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f609000cc20 tx=0x7f609000a060 comp rx=0 tx=0).stop 2023-06-28T20:07:50.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.804+0000 7f609f196700 1 -- 172.21.15.43:0/4181133539 shutdown_connections 2023-06-28T20:07:50.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.804+0000 7f609f196700 1 --2- 172.21.15.43:0/4181133539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60980af060 0x7f60980c5e30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.804+0000 7f609f196700 1 --2- 172.21.15.43:0/4181133539 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f609806ccc0 0x7f609815dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.804+0000 7f609f196700 1 --2- 172.21.15.43:0/4181133539 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f607c06ad10 0x7f607c06d1c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.804+0000 7f609f196700 1 --2- 172.21.15.43:0/4181133539 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f60980aec90 0x7f609815ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.804+0000 7f609f196700 1 -- 172.21.15.43:0/4181133539 >> 172.21.15.43:0/4181133539 conn(0x7f609806b000 msgr2=0x7f609806d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:50.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.804+0000 7f609f196700 1 -- 172.21.15.43:0/4181133539 shutdown_connections 2023-06-28T20:07:50.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.804+0000 7f609f196700 1 -- 172.21.15.43:0/4181133539 wait complete. 2023-06-28T20:07:50.819 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-28T20:07:50.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.945+0000 7f32baa55700 1 Processor -- start 2023-06-28T20:07:50.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.945+0000 7f32baa55700 1 -- start start 2023-06-28T20:07:50.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.946+0000 7f32baa55700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f32b413fa20 0x7f32b40c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:50.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.946+0000 7f32baa55700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f32b415a890 0x7f32b40c5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:50.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.946+0000 7f32baa55700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b415de80 con 0x7f32b415e310 2023-06-28T20:07:50.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.946+0000 7f32baa55700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b406a210 con 0x7f32b415a890 2023-06-28T20:07:50.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.946+0000 7f32baa55700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b415adc0 con 0x7f32b413fa20 2023-06-28T20:07:50.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.946+0000 7f32b37fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f32b415a890 0x7f32b40c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:50.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.946+0000 7f32b8ff2700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f32b415e310 0x7f32b415e760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43120/0 (socket says 172.21.15.43:43120) 2023-06-28T20:07:50.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.946+0000 7f32b8ff2700 1 -- 172.21.15.43:0/721489331 learned_addr learned my addr 172.21.15.43:0/721489331 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:50.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.946+0000 7f32b3fff700 1 --2- 172.21.15.43:0/721489331 >> v2:172.21.15.178:3300/0 conn(0x7f32b413fa20 0x7f32b40c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:50.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.946+0000 7f32b37fe700 1 -- 172.21.15.43:0/721489331 >> v2:172.21.15.178:3300/0 conn(0x7f32b413fa20 msgr2=0x7f32b40c4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:50.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.946+0000 7f32b37fe700 1 --2- 172.21.15.43:0/721489331 >> v2:172.21.15.178:3300/0 conn(0x7f32b413fa20 0x7f32b40c4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.946+0000 7f32b37fe700 1 -- 172.21.15.43:0/721489331 >> v1:172.21.15.43:6789/0 conn(0x7f32b415e310 legacy=0x7f32b415e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:50.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.946+0000 7f32b37fe700 1 -- 172.21.15.43:0/721489331 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b415b0a0 con 0x7f32b415a890 2023-06-28T20:07:50.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.946+0000 7f32b2ffd700 1 -- 172.21.15.43:0/721489331 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3367015282 0 0) 0x7f32b415de80 con 0x7f32b415e310 2023-06-28T20:07:50.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.946+0000 7f32b3fff700 1 --2- 172.21.15.43:0/721489331 >> v2:172.21.15.178:3300/0 conn(0x7f32b413fa20 0x7f32b40c4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:50.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.946+0000 7f32b37fe700 1 --2- 172.21.15.43:0/721489331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b415a890 0x7f32b40c5360 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f32b40b5050 tx=0x7f32a8009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c807f013e47e81 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:50.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.947+0000 7f32b2ffd700 1 -- 172.21.15.43:0/721489331 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a8005e30 con 0x7f32b415a890 2023-06-28T20:07:50.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.947+0000 7f32b2ffd700 1 -- 172.21.15.43:0/721489331 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a8006030 con 0x7f32b415a890 2023-06-28T20:07:50.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.947+0000 7f32b2ffd700 1 -- 172.21.15.43:0/721489331 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a80063c0 con 0x7f32b415a890 2023-06-28T20:07:50.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.947+0000 7f32baa55700 1 -- 172.21.15.43:0/721489331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b415a890 msgr2=0x7f32b40c5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:50.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.947+0000 7f32baa55700 1 --2- 172.21.15.43:0/721489331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b415a890 0x7f32b40c5360 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f32b40b5050 tx=0x7f32a8009b00 comp rx=0 tx=0).stop 2023-06-28T20:07:50.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.947+0000 7f32baa55700 1 -- 172.21.15.43:0/721489331 shutdown_connections 2023-06-28T20:07:50.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.947+0000 7f32baa55700 1 --2- 172.21.15.43:0/721489331 >> v2:172.21.15.178:3300/0 conn(0x7f32b413fa20 0x7f32b40c4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.947+0000 7f32baa55700 1 --2- 172.21.15.43:0/721489331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b415a890 0x7f32b40c5360 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.947+0000 7f32baa55700 1 -- 172.21.15.43:0/721489331 >> 172.21.15.43:0/721489331 conn(0x7f32b40cfa80 msgr2=0x7f32b40cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:50.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.947+0000 7f32baa55700 1 -- 172.21.15.43:0/721489331 shutdown_connections 2023-06-28T20:07:50.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.947+0000 7f32baa55700 1 -- 172.21.15.43:0/721489331 wait complete. 2023-06-28T20:07:50.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.948+0000 7f32baa55700 1 Processor -- start 2023-06-28T20:07:50.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.949+0000 7f32baa55700 1 -- start start 2023-06-28T20:07:50.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.949+0000 7f32baa55700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32b413fa20 0x7f32b406ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:50.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.949+0000 7f32baa55700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b415a890 0x7f32b40c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:50.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.949+0000 7f32baa55700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32b415e310 0x7f32b40ce040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:50.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.949+0000 7f32baa55700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b40cdbb0 con 0x7f32b415a890 2023-06-28T20:07:50.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.949+0000 7f32baa55700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b40ccfa0 con 0x7f32b413fa20 2023-06-28T20:07:50.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.949+0000 7f32baa55700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b40cd280 con 0x7f32b415e310 2023-06-28T20:07:50.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.949+0000 7f32b37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b415a890 0x7f32b40c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:50.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.949+0000 7f32b37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b415a890 0x7f32b40c5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53938/0 (socket says 172.21.15.43:53938) 2023-06-28T20:07:50.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.949+0000 7f32b37fe700 1 -- 172.21.15.43:0/1942010290 learned_addr learned my addr 172.21.15.43:0/1942010290 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:50.958 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.950+0000 7f32b3fff700 1 --2- 172.21.15.43:0/1942010290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32b413fa20 0x7f32b406ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:50.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.950+0000 7f32b8ff2700 1 --2- 172.21.15.43:0/1942010290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32b415e310 0x7f32b40ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:50.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.950+0000 7f32b37fe700 1 -- 172.21.15.43:0/1942010290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32b415e310 msgr2=0x7f32b40ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:50.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.950+0000 7f32b37fe700 1 --2- 172.21.15.43:0/1942010290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32b415e310 0x7f32b40ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.950+0000 7f32b37fe700 1 -- 172.21.15.43:0/1942010290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32b413fa20 msgr2=0x7f32b406ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:50.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.950+0000 7f32b37fe700 1 --2- 172.21.15.43:0/1942010290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32b413fa20 0x7f32b406ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:50.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.950+0000 7f32b37fe700 1 -- 172.21.15.43:0/1942010290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a8009430 con 0x7f32b415a890 2023-06-28T20:07:50.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.950+0000 7f32b37fe700 1 --2- 172.21.15.43:0/1942010290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b415a890 0x7f32b40c5e80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009b40 tx=0x7f32a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:50.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.950+0000 7f32b0ff9700 1 -- 172.21.15.43:0/1942010290 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a8004040 con 0x7f32b415a890 2023-06-28T20:07:50.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.950+0000 7f32b0ff9700 1 -- 172.21.15.43:0/1942010290 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a8020070 con 0x7f32b415a890 2023-06-28T20:07:50.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.950+0000 7f32baa55700 1 -- 172.21.15.43:0/1942010290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b40cd560 con 0x7f32b415a890 2023-06-28T20:07:50.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.950+0000 7f32b0ff9700 1 -- 172.21.15.43:0/1942010290 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a80043e0 con 0x7f32b415a890 2023-06-28T20:07:50.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.950+0000 7f32baa55700 1 -- 172.21.15.43:0/1942010290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b40704c0 con 0x7f32b415a890 2023-06-28T20:07:50.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.951+0000 7f32baa55700 1 -- 172.21.15.43:0/1942010290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b40cdd60 con 0x7f32b415a890 2023-06-28T20:07:50.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.952+0000 7f32b0ff9700 1 -- 172.21.15.43:0/1942010290 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f32a8022030 con 0x7f32b415a890 2023-06-28T20:07:50.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.952+0000 7f32b0ff9700 1 --2- 172.21.15.43:0/1942010290 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f32a006ace0 0x7f32a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:50.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.952+0000 7f32b0ff9700 1 -- 172.21.15.43:0/1942010290 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f32a808ab30 con 0x7f32b415a890 2023-06-28T20:07:50.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.952+0000 7f32b3fff700 1 --2- 172.21.15.43:0/1942010290 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f32a006ace0 0x7f32a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:50.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.953+0000 7f32b3fff700 1 --2- 172.21.15.43:0/1942010290 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f32a006ace0 0x7f32a006d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f32a4006fd0 tx=0x7f32a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:50.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:50.957+0000 7f32b0ff9700 1 -- 172.21.15.43:0/1942010290 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f32b40cdd60 con 0x7f32b415a890 2023-06-28T20:07:51.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.156+0000 7f32baa55700 1 -- 172.21.15.43:0/1942010290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f32b4070650 con 0x7f32b415a890 2023-06-28T20:07:51.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.156+0000 7f32b0ff9700 1 -- 172.21.15.43:0/1942010290 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f32a8055c70 con 0x7f32b415a890 2023-06-28T20:07:51.159 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-06-28T20:07:51.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.158+0000 7f32baa55700 1 -- 172.21.15.43:0/1942010290 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f32a006ace0 msgr2=0x7f32a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:51.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.158+0000 7f32baa55700 1 --2- 172.21.15.43:0/1942010290 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f32a006ace0 0x7f32a006d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f32a4006fd0 tx=0x7f32a4008040 comp rx=0 tx=0).stop 2023-06-28T20:07:51.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.158+0000 7f32baa55700 1 -- 172.21.15.43:0/1942010290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b415a890 msgr2=0x7f32b40c5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:51.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.158+0000 7f32baa55700 1 --2- 172.21.15.43:0/1942010290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b415a890 0x7f32b40c5e80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009b40 tx=0x7f32a8004750 comp rx=0 tx=0).stop 2023-06-28T20:07:51.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.158+0000 7f32baa55700 1 -- 172.21.15.43:0/1942010290 shutdown_connections 2023-06-28T20:07:51.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.158+0000 7f32baa55700 1 --2- 172.21.15.43:0/1942010290 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f32a006ace0 0x7f32a006d190 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.158+0000 7f32baa55700 1 --2- 172.21.15.43:0/1942010290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f32b415e310 0x7f32b40ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.158+0000 7f32baa55700 1 --2- 172.21.15.43:0/1942010290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32b413fa20 0x7f32b406ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.158+0000 7f32baa55700 1 --2- 172.21.15.43:0/1942010290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b415a890 0x7f32b40c5e80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.158+0000 7f32baa55700 1 -- 172.21.15.43:0/1942010290 >> 172.21.15.43:0/1942010290 conn(0x7f32b40cfa80 msgr2=0x7f32b406cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:51.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.158+0000 7f32baa55700 1 -- 172.21.15.43:0/1942010290 shutdown_connections 2023-06-28T20:07:51.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.158+0000 7f32baa55700 1 -- 172.21.15.43:0/1942010290 wait complete. 2023-06-28T20:07:51.173 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:07:51.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.296+0000 7f3bd1fd2700 1 Processor -- start 2023-06-28T20:07:51.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.296+0000 7f3bd1fd2700 1 -- start start 2023-06-28T20:07:51.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.297+0000 7f3bd1fd2700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3bcc071580 0x7f3bcc0ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:51.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.297+0000 7f3bd1fd2700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3bcc071950 0x7f3bcc0c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:51.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.297+0000 7f3bd1fd2700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3bcc069ef0 0x7f3bcc06a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:51.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.297+0000 7f3bd1fd2700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bcc0c9f50 con 0x7f3bcc071950 2023-06-28T20:07:51.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.297+0000 7f3bd1fd2700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bcc0cb370 con 0x7f3bcc071580 2023-06-28T20:07:51.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.297+0000 7f3bd1fd2700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc0cbbb0 con 0x7f3bcc069ef0 2023-06-28T20:07:51.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.297+0000 7f3bcaffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3bcc071950 0x7f3bcc0c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:51.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.297+0000 7f3bcaffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3bcc071950 0x7f3bcc0c9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53946/0 (socket says 172.21.15.43:53946) 2023-06-28T20:07:51.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.297+0000 7f3bcaffd700 1 -- 172.21.15.43:0/843174961 learned_addr learned my addr 172.21.15.43:0/843174961 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:51.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.297+0000 7f3bcbfff700 1 --2- 172.21.15.43:0/843174961 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3bcc069ef0 0x7f3bcc06a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:51.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.298+0000 7f3bcb7fe700 1 --2- 172.21.15.43:0/843174961 >> v2:172.21.15.178:3300/0 conn(0x7f3bcc071580 0x7f3bcc0ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:51.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.298+0000 7f3bcaffd700 1 -- 172.21.15.43:0/843174961 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3bcc069ef0 msgr2=0x7f3bcc06a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:51.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.298+0000 7f3bcaffd700 1 --2- 172.21.15.43:0/843174961 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3bcc069ef0 0x7f3bcc06a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.298+0000 7f3bcaffd700 1 -- 172.21.15.43:0/843174961 >> v2:172.21.15.178:3300/0 conn(0x7f3bcc071580 msgr2=0x7f3bcc0ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:51.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.298+0000 7f3bcaffd700 1 --2- 172.21.15.43:0/843174961 >> v2:172.21.15.178:3300/0 conn(0x7f3bcc071580 0x7f3bcc0ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.298+0000 7f3bcaffd700 1 -- 172.21.15.43:0/843174961 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc0cb5f0 con 0x7f3bcc071950 2023-06-28T20:07:51.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.298+0000 7f3bcbfff700 1 --2- 172.21.15.43:0/843174961 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3bcc069ef0 0x7f3bcc06a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:51.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.298+0000 7f3bcaffd700 1 --2- 172.21.15.43:0/843174961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bcc071950 0x7f3bcc0c9a10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc0cbda0 tx=0x7f3bc000ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228bb0a8f7c83296 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:51.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.298+0000 7f3bca7fc700 1 -- 172.21.15.43:0/843174961 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc000cb70 con 0x7f3bcc071950 2023-06-28T20:07:51.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.298+0000 7f3bca7fc700 1 -- 172.21.15.43:0/843174961 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc000ccd0 con 0x7f3bcc071950 2023-06-28T20:07:51.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.298+0000 7f3bca7fc700 1 -- 172.21.15.43:0/843174961 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0018620 con 0x7f3bcc071950 2023-06-28T20:07:51.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.298+0000 7f3bd1fd2700 1 -- 172.21.15.43:0/843174961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bcc071950 msgr2=0x7f3bcc0c9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:51.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.298+0000 7f3bd1fd2700 1 --2- 172.21.15.43:0/843174961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bcc071950 0x7f3bcc0c9a10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc0cbda0 tx=0x7f3bc000ad70 comp rx=0 tx=0).stop 2023-06-28T20:07:51.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.299+0000 7f3bd1fd2700 1 -- 172.21.15.43:0/843174961 shutdown_connections 2023-06-28T20:07:51.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.299+0000 7f3bd1fd2700 1 --2- 172.21.15.43:0/843174961 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3bcc069ef0 0x7f3bcc06a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.299+0000 7f3bd1fd2700 1 --2- 172.21.15.43:0/843174961 >> v2:172.21.15.178:3300/0 conn(0x7f3bcc071580 0x7f3bcc0ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.299+0000 7f3bd1fd2700 1 --2- 172.21.15.43:0/843174961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bcc071950 0x7f3bcc0c9a10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.299+0000 7f3bd1fd2700 1 -- 172.21.15.43:0/843174961 >> 172.21.15.43:0/843174961 conn(0x7f3bcc06c170 msgr2=0x7f3bcc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:51.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.299+0000 7f3bd1fd2700 1 -- 172.21.15.43:0/843174961 shutdown_connections 2023-06-28T20:07:51.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.299+0000 7f3bd1fd2700 1 -- 172.21.15.43:0/843174961 wait complete. 2023-06-28T20:07:51.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.300+0000 7f3bd1fd2700 1 Processor -- start 2023-06-28T20:07:51.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.300+0000 7f3bd1fd2700 1 -- start start 2023-06-28T20:07:51.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.301+0000 7f3bd1fd2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bcc069ef0 0x7f3bcc158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:51.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.301+0000 7f3bd1fd2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3bcc071580 0x7f3bcc152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:51.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.301+0000 7f3bd1fd2700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3bcc071950 0x7f3bcc0b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:51.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.301+0000 7f3bd1fd2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc159220 con 0x7f3bcc069ef0 2023-06-28T20:07:51.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.301+0000 7f3bd1fd2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc157a30 con 0x7f3bcc071580 2023-06-28T20:07:51.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.301+0000 7f3bd1fd2700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc157f60 con 0x7f3bcc071950 2023-06-28T20:07:51.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.301+0000 7f3bcb7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bcc069ef0 0x7f3bcc158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:51.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.301+0000 7f3bcb7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bcc069ef0 0x7f3bcc158ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53950/0 (socket says 172.21.15.43:53950) 2023-06-28T20:07:51.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.301+0000 7f3bcb7fe700 1 -- 172.21.15.43:0/202542684 learned_addr learned my addr 172.21.15.43:0/202542684 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:51.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.302+0000 7f3bcaffd700 1 --2- 172.21.15.43:0/202542684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3bcc071580 0x7f3bcc152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:51.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.302+0000 7f3bcbfff700 1 --2- 172.21.15.43:0/202542684 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3bcc071950 0x7f3bcc0b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:51.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.302+0000 7f3bcb7fe700 1 -- 172.21.15.43:0/202542684 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3bcc071950 msgr2=0x7f3bcc0b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:51.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.302+0000 7f3bcb7fe700 1 --2- 172.21.15.43:0/202542684 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3bcc071950 0x7f3bcc0b4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.302+0000 7f3bcb7fe700 1 -- 172.21.15.43:0/202542684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3bcc071580 msgr2=0x7f3bcc152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.302+0000 7f3bcb7fe700 1 --2- 172.21.15.43:0/202542684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3bcc071580 0x7f3bcc152640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.302+0000 7f3bcb7fe700 1 -- 172.21.15.43:0/202542684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc000a6f0 con 0x7f3bcc069ef0 2023-06-28T20:07:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.302+0000 7f3bcb7fe700 1 --2- 172.21.15.43:0/202542684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bcc069ef0 0x7f3bcc158ce0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc010730 tx=0x7f3bbc010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.302+0000 7f3bb3fff700 1 -- 172.21.15.43:0/202542684 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc0118c0 con 0x7f3bcc069ef0 2023-06-28T20:07:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.302+0000 7f3bb3fff700 1 -- 172.21.15.43:0/202542684 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc011a20 con 0x7f3bcc069ef0 2023-06-28T20:07:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.302+0000 7f3bd1fd2700 1 -- 172.21.15.43:0/202542684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bcc158200 con 0x7f3bcc069ef0 2023-06-28T20:07:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.302+0000 7f3bb3fff700 1 -- 172.21.15.43:0/202542684 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc011dd0 con 0x7f3bcc069ef0 2023-06-28T20:07:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.302+0000 7f3bd1fd2700 1 -- 172.21.15.43:0/202542684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bcc1587c0 con 0x7f3bcc069ef0 2023-06-28T20:07:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.303+0000 7f3bd1fd2700 1 -- 172.21.15.43:0/202542684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bcc158b30 con 0x7f3bcc069ef0 2023-06-28T20:07:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.304+0000 7f3bb3fff700 1 -- 172.21.15.43:0/202542684 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3bbc019430 con 0x7f3bcc069ef0 2023-06-28T20:07:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.304+0000 7f3bb3fff700 1 --2- 172.21.15.43:0/202542684 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f3bb806ace0 0x7f3bb806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.304+0000 7f3bb3fff700 1 -- 172.21.15.43:0/202542684 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3bbc08b4c0 con 0x7f3bcc069ef0 2023-06-28T20:07:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.304+0000 7f3bcaffd700 1 --2- 172.21.15.43:0/202542684 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f3bb806ace0 0x7f3bb806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.304+0000 7f3bcaffd700 1 --2- 172.21.15.43:0/202542684 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f3bb806ace0 0x7f3bb806d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000afd0 tx=0x7f3bc001a040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:51.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.309+0000 7f3bb3fff700 1 -- 172.21.15.43:0/202542684 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3bbc0564f0 con 0x7f3bcc069ef0 2023-06-28T20:07:51.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.509+0000 7f3bd1fd2700 1 -- 172.21.15.43:0/202542684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bcc158950 con 0x7f3bcc069ef0 2023-06-28T20:07:51.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.510+0000 7f3bb3fff700 1 -- 172.21.15.43:0/202542684 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18221 (secure 0 0 0) 0x7f3bbc059b10 con 0x7f3bcc069ef0 2023-06-28T20:07:51.514 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:51.514 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:07:22.377248+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:07:18.624759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-28T20:07:19.760017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:07:51.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.513+0000 7f3bd1fd2700 1 -- 172.21.15.43:0/202542684 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f3bb806ace0 msgr2=0x7f3bb806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:51.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.513+0000 7f3bd1fd2700 1 --2- 172.21.15.43:0/202542684 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f3bb806ace0 0x7f3bb806d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000afd0 tx=0x7f3bc001a040 comp rx=0 tx=0).stop 2023-06-28T20:07:51.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.513+0000 7f3bd1fd2700 1 -- 172.21.15.43:0/202542684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bcc069ef0 msgr2=0x7f3bcc158ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:51.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.513+0000 7f3bd1fd2700 1 --2- 172.21.15.43:0/202542684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bcc069ef0 0x7f3bcc158ce0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc010730 tx=0x7f3bbc010a40 comp rx=0 tx=0).stop 2023-06-28T20:07:51.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.513+0000 7f3bd1fd2700 1 -- 172.21.15.43:0/202542684 shutdown_connections 2023-06-28T20:07:51.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.513+0000 7f3bd1fd2700 1 --2- 172.21.15.43:0/202542684 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f3bb806ace0 0x7f3bb806d190 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.513+0000 7f3bd1fd2700 1 --2- 172.21.15.43:0/202542684 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3bcc071950 0x7f3bcc0b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.513+0000 7f3bd1fd2700 1 --2- 172.21.15.43:0/202542684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bcc069ef0 0x7f3bcc158ce0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.513+0000 7f3bd1fd2700 1 --2- 172.21.15.43:0/202542684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3bcc071580 0x7f3bcc152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.513+0000 7f3bd1fd2700 1 -- 172.21.15.43:0/202542684 >> 172.21.15.43:0/202542684 conn(0x7f3bcc06c170 msgr2=0x7f3bcc0cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:51.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.513+0000 7f3bd1fd2700 1 -- 172.21.15.43:0/202542684 shutdown_connections 2023-06-28T20:07:51.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.513+0000 7f3bd1fd2700 1 -- 172.21.15.43:0/202542684 wait complete. 2023-06-28T20:07:51.528 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-28T20:07:13.580969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-28T20:07:14.038358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-28T20:07:18.624759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-28T20:07:19.760017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-28T20:07:51.529 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-28T20:07:51.656 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.653+0000 7f1f35b26700 1 Processor -- start 2023-06-28T20:07:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.653+0000 7f1f35b26700 1 -- start start 2023-06-28T20:07:51.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.654+0000 7f1f35b26700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1f301425a0 0x7f1f3015a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:51.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.654+0000 7f1f35b26700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1f300c9900 0x7f1f300c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:51.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.654+0000 7f1f35b26700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f300cd590 con 0x7f1f30159e80 2023-06-28T20:07:51.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.654+0000 7f1f35b26700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f3015a940 con 0x7f1f300c9900 2023-06-28T20:07:51.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.654+0000 7f1f35b26700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f300ca240 con 0x7f1f301425a0 2023-06-28T20:07:51.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.654+0000 7f1f2ffff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1f300c9900 0x7f1f300c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:51.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.654+0000 7f1f2f7fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1f30159e80 0x7f1f300cce80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43144/0 (socket says 172.21.15.43:43144) 2023-06-28T20:07:51.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.654+0000 7f1f2f7fe700 1 -- 172.21.15.43:0/13624529 learned_addr learned my addr 172.21.15.43:0/13624529 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:51.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.654+0000 7f1f2effd700 1 --2- 172.21.15.43:0/13624529 >> v2:172.21.15.178:3300/0 conn(0x7f1f301425a0 0x7f1f3015a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:51.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.655+0000 7f1f2ffff700 1 -- 172.21.15.43:0/13624529 >> v2:172.21.15.178:3300/0 conn(0x7f1f301425a0 msgr2=0x7f1f3015a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:51.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.655+0000 7f1f2ffff700 1 --2- 172.21.15.43:0/13624529 >> v2:172.21.15.178:3300/0 conn(0x7f1f301425a0 0x7f1f3015a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.655+0000 7f1f2ffff700 1 -- 172.21.15.43:0/13624529 >> v1:172.21.15.43:6789/0 conn(0x7f1f30159e80 legacy=0x7f1f300cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:51.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.655+0000 7f1f2ffff700 1 -- 172.21.15.43:0/13624529 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f3006a090 con 0x7f1f300c9900 2023-06-28T20:07:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.655+0000 7f1f2e7fc700 1 -- 172.21.15.43:0/13624529 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3465446676 0 0) 0x7f1f300cd590 con 0x7f1f30159e80 2023-06-28T20:07:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.655+0000 7f1f2ffff700 1 --2- 172.21.15.43:0/13624529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f300c9900 0x7f1f300c9d00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1f300aecc0 tx=0x7f1f2800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f533e67beff5e17f server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.655+0000 7f1f2e7fc700 1 -- 172.21.15.43:0/13624529 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2800e070 con 0x7f1f300c9900 2023-06-28T20:07:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.655+0000 7f1f2e7fc700 1 -- 172.21.15.43:0/13624529 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2800bb20 con 0x7f1f300c9900 2023-06-28T20:07:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.655+0000 7f1f2e7fc700 1 -- 172.21.15.43:0/13624529 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f28012430 con 0x7f1f300c9900 2023-06-28T20:07:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.655+0000 7f1f35b26700 1 -- 172.21.15.43:0/13624529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f300c9900 msgr2=0x7f1f300c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.655+0000 7f1f35b26700 1 --2- 172.21.15.43:0/13624529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f300c9900 0x7f1f300c9d00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1f300aecc0 tx=0x7f1f2800b330 comp rx=0 tx=0).stop 2023-06-28T20:07:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.655+0000 7f1f35b26700 1 -- 172.21.15.43:0/13624529 shutdown_connections 2023-06-28T20:07:51.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.655+0000 7f1f35b26700 1 --2- 172.21.15.43:0/13624529 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f300c9900 0x7f1f300c9d00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.655+0000 7f1f35b26700 1 --2- 172.21.15.43:0/13624529 >> v2:172.21.15.178:3300/0 conn(0x7f1f301425a0 0x7f1f3015a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.655+0000 7f1f35b26700 1 -- 172.21.15.43:0/13624529 >> 172.21.15.43:0/13624529 conn(0x7f1f300b1a50 msgr2=0x7f1f300cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:51.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.656+0000 7f1f35b26700 1 -- 172.21.15.43:0/13624529 shutdown_connections 2023-06-28T20:07:51.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.656+0000 7f1f35b26700 1 -- 172.21.15.43:0/13624529 wait complete. 2023-06-28T20:07:51.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.657+0000 7f1f35b26700 1 Processor -- start 2023-06-28T20:07:51.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.657+0000 7f1f35b26700 1 -- start start 2023-06-28T20:07:51.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.658+0000 7f1f35b26700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f300c9900 0x7f1f300af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:51.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.658+0000 7f1f35b26700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f301425a0 0x7f1f300b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:51.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.658+0000 7f1f35b26700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f30159e80 0x7f1f30156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:51.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.658+0000 7f1f35b26700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f300b3f60 con 0x7f1f30159e80 2023-06-28T20:07:51.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.658+0000 7f1f35b26700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f300b30b0 con 0x7f1f300c9900 2023-06-28T20:07:51.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.658+0000 7f1f35b26700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f300b3350 con 0x7f1f301425a0 2023-06-28T20:07:51.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.658+0000 7f1f2ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f30159e80 0x7f1f30156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:51.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.658+0000 7f1f2ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f30159e80 0x7f1f30156380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53968/0 (socket says 172.21.15.43:53968) 2023-06-28T20:07:51.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.658+0000 7f1f2ffff700 1 -- 172.21.15.43:0/1770283897 learned_addr learned my addr 172.21.15.43:0/1770283897 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:51.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.658+0000 7f1f2f7fe700 1 --2- 172.21.15.43:0/1770283897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f300c9900 0x7f1f300af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:51.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.658+0000 7f1f2effd700 1 --2- 172.21.15.43:0/1770283897 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f301425a0 0x7f1f300b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:51.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.658+0000 7f1f2ffff700 1 -- 172.21.15.43:0/1770283897 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f301425a0 msgr2=0x7f1f300b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:51.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.658+0000 7f1f2ffff700 1 --2- 172.21.15.43:0/1770283897 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f301425a0 0x7f1f300b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.658+0000 7f1f2ffff700 1 -- 172.21.15.43:0/1770283897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f300c9900 msgr2=0x7f1f300af220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:51.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.658+0000 7f1f2ffff700 1 --2- 172.21.15.43:0/1770283897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f300c9900 0x7f1f300af220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.658+0000 7f1f2ffff700 1 -- 172.21.15.43:0/1770283897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f28009a70 con 0x7f1f30159e80 2023-06-28T20:07:51.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.658+0000 7f1f2effd700 1 --2- 172.21.15.43:0/1770283897 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f301425a0 0x7f1f300b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:51.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.658+0000 7f1f2ffff700 1 --2- 172.21.15.43:0/1770283897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f30159e80 0x7f1f30156380 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1f30067270 tx=0x7f1f28004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:51.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.658+0000 7f1f27fff700 1 -- 172.21.15.43:0/1770283897 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f28012430 con 0x7f1f30159e80 2023-06-28T20:07:51.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.658+0000 7f1f27fff700 1 -- 172.21.15.43:0/1770283897 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f28020070 con 0x7f1f30159e80 2023-06-28T20:07:51.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.658+0000 7f1f35b26700 1 -- 172.21.15.43:0/1770283897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f300b3630 con 0x7f1f30159e80 2023-06-28T20:07:51.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.658+0000 7f1f27fff700 1 -- 172.21.15.43:0/1770283897 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f28004280 con 0x7f1f30159e80 2023-06-28T20:07:51.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.658+0000 7f1f35b26700 1 -- 172.21.15.43:0/1770283897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f300b3bf0 con 0x7f1f30159e80 2023-06-28T20:07:51.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.659+0000 7f1f35b26700 1 -- 172.21.15.43:0/1770283897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f300b40d0 con 0x7f1f30159e80 2023-06-28T20:07:51.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.660+0000 7f1f27fff700 1 -- 172.21.15.43:0/1770283897 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1f280043e0 con 0x7f1f30159e80 2023-06-28T20:07:51.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.660+0000 7f1f27fff700 1 --2- 172.21.15.43:0/1770283897 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1f1006ace0 0x7f1f1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:51.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.660+0000 7f1f27fff700 1 -- 172.21.15.43:0/1770283897 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1f28022070 con 0x7f1f30159e80 2023-06-28T20:07:51.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.660+0000 7f1f2f7fe700 1 --2- 172.21.15.43:0/1770283897 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1f1006ace0 0x7f1f1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:51.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.661+0000 7f1f2f7fe700 1 --2- 172.21.15.43:0/1770283897 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1f1006ace0 0x7f1f1006d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1f20000f60 tx=0x7f1f20007e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:51.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.665+0000 7f1f27fff700 1 -- 172.21.15.43:0/1770283897 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1f280565c0 con 0x7f1f30159e80 2023-06-28T20:07:51.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.862+0000 7f1f35b26700 1 -- 172.21.15.43:0/1770283897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1f300728b0 con 0x7f1f30159e80 2023-06-28T20:07:51.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.862+0000 7f1f27fff700 1 -- 172.21.15.43:0/1770283897 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1f280263e0 con 0x7f1f30159e80 2023-06-28T20:07:51.866 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-06-28T20:07:51.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.864+0000 7f1f35b26700 1 -- 172.21.15.43:0/1770283897 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1f1006ace0 msgr2=0x7f1f1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:51.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.864+0000 7f1f35b26700 1 --2- 172.21.15.43:0/1770283897 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1f1006ace0 0x7f1f1006d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1f20000f60 tx=0x7f1f20007e70 comp rx=0 tx=0).stop 2023-06-28T20:07:51.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.864+0000 7f1f35b26700 1 -- 172.21.15.43:0/1770283897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f30159e80 msgr2=0x7f1f30156380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:51.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.864+0000 7f1f35b26700 1 --2- 172.21.15.43:0/1770283897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f30159e80 0x7f1f30156380 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1f30067270 tx=0x7f1f28004ad0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.864+0000 7f1f35b26700 1 -- 172.21.15.43:0/1770283897 shutdown_connections 2023-06-28T20:07:51.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.864+0000 7f1f35b26700 1 --2- 172.21.15.43:0/1770283897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f30159e80 0x7f1f30156380 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.864+0000 7f1f35b26700 1 --2- 172.21.15.43:0/1770283897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f300c9900 0x7f1f300af220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.864+0000 7f1f35b26700 1 --2- 172.21.15.43:0/1770283897 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1f1006ace0 0x7f1f1006d190 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.864+0000 7f1f35b26700 1 --2- 172.21.15.43:0/1770283897 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1f301425a0 0x7f1f300b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:51.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.864+0000 7f1f35b26700 1 -- 172.21.15.43:0/1770283897 >> 172.21.15.43:0/1770283897 conn(0x7f1f300b1a50 msgr2=0x7f1f3006c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:51.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.865+0000 7f1f35b26700 1 -- 172.21.15.43:0/1770283897 shutdown_connections 2023-06-28T20:07:51.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:51.865+0000 7f1f35b26700 1 -- 172.21.15.43:0/1770283897 wait complete. 2023-06-28T20:07:51.880 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-28T20:07:52.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.004+0000 7fcbbc40d700 1 Processor -- start 2023-06-28T20:07:52.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.004+0000 7fcbbc40d700 1 -- start start 2023-06-28T20:07:52.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.005+0000 7fcbbc40d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbb40af020 0x7fcbb4069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:52.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.005+0000 7fcbbc40d700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcbb40b1940 0x7fcbb40b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:52.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.005+0000 7fcbbc40d700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb415df10 con 0x7fcbb40cc650 2023-06-28T20:07:52.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.005+0000 7fcbbc40d700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb40aee10 con 0x7fcbb40b1940 2023-06-28T20:07:52.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.005+0000 7fcbbc40d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb40b2300 con 0x7fcbb40af020 2023-06-28T20:07:52.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.006+0000 7fcbba9aa700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcbb40b1940 0x7fcbb40b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:52.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.006+0000 7fcbb99a8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbb40af020 0x7fcbb4069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:52.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.006+0000 7fcbba1a9700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fcbb40cc650 0x7fcbb415e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:42282/0 (socket says 172.21.15.43:42282) 2023-06-28T20:07:52.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.006+0000 7fcbba1a9700 1 -- 172.21.15.43:0/2833336401 learned_addr learned my addr 172.21.15.43:0/2833336401 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:52.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.006+0000 7fcbb99a8700 1 -- 172.21.15.43:0/2833336401 >> v2:172.21.15.178:3300/0 conn(0x7fcbb40b1940 msgr2=0x7fcbb40b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:52.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.006+0000 7fcbb99a8700 1 --2- 172.21.15.43:0/2833336401 >> v2:172.21.15.178:3300/0 conn(0x7fcbb40b1940 0x7fcbb40b1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.006+0000 7fcbb99a8700 1 -- 172.21.15.43:0/2833336401 >> v1:172.21.15.178:6789/0 conn(0x7fcbb40cc650 legacy=0x7fcbb415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:52.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.006+0000 7fcbb99a8700 1 -- 172.21.15.43:0/2833336401 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb406a580 con 0x7fcbb40af020 2023-06-28T20:07:52.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.006+0000 7fcbba9aa700 1 --2- 172.21.15.43:0/2833336401 >> v2:172.21.15.178:3300/0 conn(0x7fcbb40b1940 0x7fcbb40b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:52.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.006+0000 7fcbb99a8700 1 --2- 172.21.15.43:0/2833336401 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbb40af020 0x7fcbb4069ef0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009840 tx=0x7fcbb0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=144c98f090679d7e server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:52.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.006+0000 7fcbb91a7700 1 -- 172.21.15.43:0/2833336401 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0011830 con 0x7fcbb40af020 2023-06-28T20:07:52.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.006+0000 7fcbb91a7700 1 -- 172.21.15.43:0/2833336401 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb0011990 con 0x7fcbb40af020 2023-06-28T20:07:52.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.006+0000 7fcbb91a7700 1 -- 172.21.15.43:0/2833336401 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0011d20 con 0x7fcbb40af020 2023-06-28T20:07:52.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.007+0000 7fcbbc40d700 1 -- 172.21.15.43:0/2833336401 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbb40af020 msgr2=0x7fcbb4069ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:52.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.007+0000 7fcbbc40d700 1 --2- 172.21.15.43:0/2833336401 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbb40af020 0x7fcbb4069ef0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009840 tx=0x7fcbb0009b50 comp rx=0 tx=0).stop 2023-06-28T20:07:52.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.007+0000 7fcbbc40d700 1 -- 172.21.15.43:0/2833336401 shutdown_connections 2023-06-28T20:07:52.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.007+0000 7fcbbc40d700 1 --2- 172.21.15.43:0/2833336401 >> v2:172.21.15.178:3300/0 conn(0x7fcbb40b1940 0x7fcbb40b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.007+0000 7fcbbc40d700 1 --2- 172.21.15.43:0/2833336401 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbb40af020 0x7fcbb4069ef0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.007+0000 7fcbbc40d700 1 -- 172.21.15.43:0/2833336401 >> 172.21.15.43:0/2833336401 conn(0x7fcbb406b000 msgr2=0x7fcbb406b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:52.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.007+0000 7fcbbc40d700 1 -- 172.21.15.43:0/2833336401 shutdown_connections 2023-06-28T20:07:52.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.007+0000 7fcbbc40d700 1 -- 172.21.15.43:0/2833336401 wait complete. 2023-06-28T20:07:52.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.008+0000 7fcbbc40d700 1 Processor -- start 2023-06-28T20:07:52.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.008+0000 7fcbbc40d700 1 -- start start 2023-06-28T20:07:52.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.009+0000 7fcbbc40d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbb40af020 0x7fcbb41502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:52.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.009+0000 7fcbbc40d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbb40b1940 0x7fcbb40b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:52.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.009+0000 7fcbbc40d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb40cc650 0x7fcbb40b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:52.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.009+0000 7fcbbc40d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb40c3850 con 0x7fcbb40af020 2023-06-28T20:07:52.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.009+0000 7fcbbc40d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb40c9a70 con 0x7fcbb40cc650 2023-06-28T20:07:52.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.009+0000 7fcbbc40d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb40c91d0 con 0x7fcbb40b1940 2023-06-28T20:07:52.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.009+0000 7fcbba1a9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbb40af020 0x7fcbb41502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:52.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.009+0000 7fcbba1a9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbb40af020 0x7fcbb41502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53978/0 (socket says 172.21.15.43:53978) 2023-06-28T20:07:52.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.009+0000 7fcbba1a9700 1 -- 172.21.15.43:0/3542411081 learned_addr learned my addr 172.21.15.43:0/3542411081 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:52.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.010+0000 7fcbba9aa700 1 --2- 172.21.15.43:0/3542411081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb40cc650 0x7fcbb40b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:52.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.010+0000 7fcbb99a8700 1 --2- 172.21.15.43:0/3542411081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbb40b1940 0x7fcbb40b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:52.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.010+0000 7fcbba1a9700 1 -- 172.21.15.43:0/3542411081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbb40b1940 msgr2=0x7fcbb40b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:52.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.010+0000 7fcbba1a9700 1 --2- 172.21.15.43:0/3542411081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbb40b1940 0x7fcbb40b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.010+0000 7fcbba1a9700 1 -- 172.21.15.43:0/3542411081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb40cc650 msgr2=0x7fcbb40b09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:52.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.010+0000 7fcbba1a9700 1 --2- 172.21.15.43:0/3542411081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb40cc650 0x7fcbb40b09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.010+0000 7fcbba1a9700 1 -- 172.21.15.43:0/3542411081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac009a70 con 0x7fcbb40af020 2023-06-28T20:07:52.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.010+0000 7fcbba9aa700 1 --2- 172.21.15.43:0/3542411081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb40cc650 0x7fcbb40b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:52.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.010+0000 7fcbb99a8700 1 --2- 172.21.15.43:0/3542411081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbb40b1940 0x7fcbb40b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:52.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.010+0000 7fcbba1a9700 1 --2- 172.21.15.43:0/3542411081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbb40af020 0x7fcbb41502b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcba400c460 tx=0x7fcba400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:52.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.010+0000 7fcbaaffd700 1 -- 172.21.15.43:0/3542411081 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba4005d70 con 0x7fcbb40af020 2023-06-28T20:07:52.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.010+0000 7fcbaaffd700 1 -- 172.21.15.43:0/3542411081 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba40070d0 con 0x7fcbb40af020 2023-06-28T20:07:52.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.010+0000 7fcbbc40d700 1 -- 172.21.15.43:0/3542411081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb00094a0 con 0x7fcbb40af020 2023-06-28T20:07:52.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.010+0000 7fcbaaffd700 1 -- 172.21.15.43:0/3542411081 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba4007460 con 0x7fcbb40af020 2023-06-28T20:07:52.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.010+0000 7fcbbc40d700 1 -- 172.21.15.43:0/3542411081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb40c9790 con 0x7fcbb40af020 2023-06-28T20:07:52.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.011+0000 7fcbbc40d700 1 -- 172.21.15.43:0/3542411081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb40728b0 con 0x7fcbb40af020 2023-06-28T20:07:52.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.011+0000 7fcbaaffd700 1 -- 172.21.15.43:0/3542411081 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcba40075c0 con 0x7fcbb40af020 2023-06-28T20:07:52.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.012+0000 7fcbaaffd700 1 --2- 172.21.15.43:0/3542411081 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fcb9806ace0 0x7fcb9806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:52.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.012+0000 7fcbaaffd700 1 -- 172.21.15.43:0/3542411081 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcba4088950 con 0x7fcbb40af020 2023-06-28T20:07:52.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.012+0000 7fcbb99a8700 1 --2- 172.21.15.43:0/3542411081 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fcb9806ace0 0x7fcb9806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:52.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.012+0000 7fcbb99a8700 1 --2- 172.21.15.43:0/3542411081 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fcb9806ace0 0x7fcb9806d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4159070 tx=0x7fcbb0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:52.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.017+0000 7fcbaaffd700 1 -- 172.21.15.43:0/3542411081 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcba4053970 con 0x7fcbb40af020 2023-06-28T20:07:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.217+0000 7fcbbc40d700 1 -- 172.21.15.43:0/3542411081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcbb40c3620 con 0x7fcbb40af020 2023-06-28T20:07:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.217+0000 7fcbaaffd700 1 -- 172.21.15.43:0/3542411081 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcba4017070 con 0x7fcbb40af020 2023-06-28T20:07:52.220 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-06-28T20:07:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.219+0000 7fcbbc40d700 1 -- 172.21.15.43:0/3542411081 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fcb9806ace0 msgr2=0x7fcb9806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.219+0000 7fcbbc40d700 1 --2- 172.21.15.43:0/3542411081 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fcb9806ace0 0x7fcb9806d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4159070 tx=0x7fcbb0008030 comp rx=0 tx=0).stop 2023-06-28T20:07:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.219+0000 7fcbbc40d700 1 -- 172.21.15.43:0/3542411081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbb40af020 msgr2=0x7fcbb41502b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.219+0000 7fcbbc40d700 1 --2- 172.21.15.43:0/3542411081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbb40af020 0x7fcbb41502b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcba400c460 tx=0x7fcba400c820 comp rx=0 tx=0).stop 2023-06-28T20:07:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.219+0000 7fcbbc40d700 1 -- 172.21.15.43:0/3542411081 shutdown_connections 2023-06-28T20:07:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.219+0000 7fcbbc40d700 1 --2- 172.21.15.43:0/3542411081 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcbb40cc650 0x7fcbb40b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.219+0000 7fcbbc40d700 1 --2- 172.21.15.43:0/3542411081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbb40af020 0x7fcbb41502b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.219+0000 7fcbbc40d700 1 --2- 172.21.15.43:0/3542411081 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fcb9806ace0 0x7fcb9806d190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.219+0000 7fcbbc40d700 1 --2- 172.21.15.43:0/3542411081 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcbb40b1940 0x7fcbb40b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.219+0000 7fcbbc40d700 1 -- 172.21.15.43:0/3542411081 >> 172.21.15.43:0/3542411081 conn(0x7fcbb406b000 msgr2=0x7fcbb40ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.219+0000 7fcbbc40d700 1 -- 172.21.15.43:0/3542411081 shutdown_connections 2023-06-28T20:07:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.219+0000 7fcbbc40d700 1 -- 172.21.15.43:0/3542411081 wait complete. 2023-06-28T20:07:52.234 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-28T20:07:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.358+0000 7fc2b6d9c700 1 Processor -- start 2023-06-28T20:07:52.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.358+0000 7fc2b6d9c700 1 -- start start 2023-06-28T20:07:52.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.359+0000 7fc2b6d9c700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc2b015cae0 0x7fc2b00cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:52.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.359+0000 7fc2b6d9c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2b00cfe00 0x7fc2b006cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:52.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.359+0000 7fc2b6d9c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b00d0680 con 0x7fc2b006d310 2023-06-28T20:07:52.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.359+0000 7fc2b6d9c700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b006bb20 con 0x7fc2b015cae0 2023-06-28T20:07:52.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.359+0000 7fc2b6d9c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b006be00 con 0x7fc2b00cfe00 2023-06-28T20:07:52.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.359+0000 7fc2b4b38700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc2b006d310 0x7fc2b006d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43172/0 (socket says 172.21.15.43:43172) 2023-06-28T20:07:52.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.359+0000 7fc2b4b38700 1 -- 172.21.15.43:0/3722589007 learned_addr learned my addr 172.21.15.43:0/3722589007 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:52.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.360+0000 7fc2b5339700 1 --2- 172.21.15.43:0/3722589007 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2b00cfe00 0x7fc2b006cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:52.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.360+0000 7fc2affff700 1 --2- 172.21.15.43:0/3722589007 >> v2:172.21.15.178:3300/0 conn(0x7fc2b015cae0 0x7fc2b00cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.360+0000 7fc2af7fe700 1 -- 172.21.15.43:0/3722589007 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953895410 0 0) 0x7fc2b00d0680 con 0x7fc2b006d310 2023-06-28T20:07:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.360+0000 7fc2af7fe700 1 -- 172.21.15.43:0/3722589007 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2a00035b0 con 0x7fc2b006d310 2023-06-28T20:07:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.360+0000 7fc2b5339700 1 -- 172.21.15.43:0/3722589007 >> v2:172.21.15.178:3300/0 conn(0x7fc2b015cae0 msgr2=0x7fc2b00cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.360+0000 7fc2b5339700 1 --2- 172.21.15.43:0/3722589007 >> v2:172.21.15.178:3300/0 conn(0x7fc2b015cae0 0x7fc2b00cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.360+0000 7fc2b5339700 1 -- 172.21.15.43:0/3722589007 >> v1:172.21.15.43:6789/0 conn(0x7fc2b006d310 legacy=0x7fc2b006d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.360+0000 7fc2b5339700 1 -- 172.21.15.43:0/3722589007 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b006c0e0 con 0x7fc2b00cfe00 2023-06-28T20:07:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.360+0000 7fc2affff700 1 --2- 172.21.15.43:0/3722589007 >> v2:172.21.15.178:3300/0 conn(0x7fc2b015cae0 0x7fc2b00cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.360+0000 7fc2b5339700 1 --2- 172.21.15.43:0/3722589007 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2b00cfe00 0x7fc2b006cbc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8009d90 tx=0x7fc2a800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=840c77d20ced6bb2 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:52.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.360+0000 7fc2af7fe700 1 -- 172.21.15.43:0/3722589007 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a800e070 con 0x7fc2b00cfe00 2023-06-28T20:07:52.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.360+0000 7fc2af7fe700 1 -- 172.21.15.43:0/3722589007 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a800bb20 con 0x7fc2b00cfe00 2023-06-28T20:07:52.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.360+0000 7fc2af7fe700 1 -- 172.21.15.43:0/3722589007 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a800be90 con 0x7fc2b00cfe00 2023-06-28T20:07:52.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.361+0000 7fc2b6d9c700 1 -- 172.21.15.43:0/3722589007 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2b00cfe00 msgr2=0x7fc2b006cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:52.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.361+0000 7fc2b6d9c700 1 --2- 172.21.15.43:0/3722589007 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2b00cfe00 0x7fc2b006cbc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8009d90 tx=0x7fc2a800b330 comp rx=0 tx=0).stop 2023-06-28T20:07:52.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.361+0000 7fc2b6d9c700 1 -- 172.21.15.43:0/3722589007 shutdown_connections 2023-06-28T20:07:52.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.361+0000 7fc2b6d9c700 1 --2- 172.21.15.43:0/3722589007 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2b00cfe00 0x7fc2b006cbc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.361+0000 7fc2b6d9c700 1 --2- 172.21.15.43:0/3722589007 >> v2:172.21.15.178:3300/0 conn(0x7fc2b015cae0 0x7fc2b00cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.361+0000 7fc2b6d9c700 1 -- 172.21.15.43:0/3722589007 >> 172.21.15.43:0/3722589007 conn(0x7fc2b00664b0 msgr2=0x7fc2b00d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:52.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.361+0000 7fc2b6d9c700 1 -- 172.21.15.43:0/3722589007 shutdown_connections 2023-06-28T20:07:52.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.361+0000 7fc2b6d9c700 1 -- 172.21.15.43:0/3722589007 wait complete. 2023-06-28T20:07:52.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.362+0000 7fc2b6d9c700 1 Processor -- start 2023-06-28T20:07:52.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.362+0000 7fc2b6d9c700 1 -- start start 2023-06-28T20:07:52.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.363+0000 7fc2b6d9c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2b006d310 0x7fc2b01f2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:52.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.363+0000 7fc2b6d9c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2b00cfe00 0x7fc2b015a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:52.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.363+0000 7fc2b6d9c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2b015cae0 0x7fc2b00cb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:52.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.363+0000 7fc2b6d9c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b00cc610 con 0x7fc2b015cae0 2023-06-28T20:07:52.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.363+0000 7fc2b6d9c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b015ae50 con 0x7fc2b006d310 2023-06-28T20:07:52.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.363+0000 7fc2b6d9c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b00cb750 con 0x7fc2b00cfe00 2023-06-28T20:07:52.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.363+0000 7fc2b5339700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2b015cae0 0x7fc2b00cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.363+0000 7fc2b5339700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2b015cae0 0x7fc2b00cb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53996/0 (socket says 172.21.15.43:53996) 2023-06-28T20:07:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.363+0000 7fc2b5339700 1 -- 172.21.15.43:0/1647351537 learned_addr learned my addr 172.21.15.43:0/1647351537 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.363+0000 7fc2affff700 1 --2- 172.21.15.43:0/1647351537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2b00cfe00 0x7fc2b015a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.363+0000 7fc2b4b38700 1 --2- 172.21.15.43:0/1647351537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2b006d310 0x7fc2b01f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.363+0000 7fc2b5339700 1 -- 172.21.15.43:0/1647351537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2b00cfe00 msgr2=0x7fc2b015a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.363+0000 7fc2b5339700 1 --2- 172.21.15.43:0/1647351537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2b00cfe00 0x7fc2b015a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.364+0000 7fc2b5339700 1 -- 172.21.15.43:0/1647351537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2b006d310 msgr2=0x7fc2b01f2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:52.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.364+0000 7fc2b5339700 1 --2- 172.21.15.43:0/1647351537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2b006d310 0x7fc2b01f2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.364+0000 7fc2b5339700 1 -- 172.21.15.43:0/1647351537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29c0094a0 con 0x7fc2b015cae0 2023-06-28T20:07:52.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.364+0000 7fc2b5339700 1 --2- 172.21.15.43:0/1647351537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2b015cae0 0x7fc2b00cb210 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8014040 tx=0x7fc2a8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:52.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.364+0000 7fc2ad7fa700 1 -- 172.21.15.43:0/1647351537 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8004800 con 0x7fc2b015cae0 2023-06-28T20:07:52.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.364+0000 7fc2ad7fa700 1 -- 172.21.15.43:0/1647351537 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a8020070 con 0x7fc2b015cae0 2023-06-28T20:07:52.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.364+0000 7fc2b6d9c700 1 -- 172.21.15.43:0/1647351537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a8009a70 con 0x7fc2b015cae0 2023-06-28T20:07:52.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.364+0000 7fc2ad7fa700 1 -- 172.21.15.43:0/1647351537 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a80276c0 con 0x7fc2b015cae0 2023-06-28T20:07:52.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.364+0000 7fc2b6d9c700 1 -- 172.21.15.43:0/1647351537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b00cbce0 con 0x7fc2b015cae0 2023-06-28T20:07:52.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.365+0000 7fc2b6d9c700 1 -- 172.21.15.43:0/1647351537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b00cc7c0 con 0x7fc2b015cae0 2023-06-28T20:07:52.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.365+0000 7fc2ad7fa700 1 -- 172.21.15.43:0/1647351537 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc2a8012430 con 0x7fc2b015cae0 2023-06-28T20:07:52.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.366+0000 7fc2ad7fa700 1 --2- 172.21.15.43:0/1647351537 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc29406ace0 0x7fc29406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:52.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.366+0000 7fc2ad7fa700 1 -- 172.21.15.43:0/1647351537 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc2a8022070 con 0x7fc2b015cae0 2023-06-28T20:07:52.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.366+0000 7fc2b4b38700 1 --2- 172.21.15.43:0/1647351537 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc29406ace0 0x7fc29406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:52.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.366+0000 7fc2b4b38700 1 --2- 172.21.15.43:0/1647351537 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc29406ace0 0x7fc29406d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4001360 tx=0x7fc2a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:52.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.370+0000 7fc2ad7fa700 1 -- 172.21.15.43:0/1647351537 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc2b00cc7c0 con 0x7fc2b015cae0 2023-06-28T20:07:52.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.568+0000 7fc2b6d9c700 1 -- 172.21.15.43:0/1647351537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc2b00728b0 con 0x7fc2b015cae0 2023-06-28T20:07:52.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.568+0000 7fc2ad7fa700 1 -- 172.21.15.43:0/1647351537 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc2a8056500 con 0x7fc2b015cae0 2023-06-28T20:07:52.572 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-06-28T20:07:52.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.570+0000 7fc2b6d9c700 1 -- 172.21.15.43:0/1647351537 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc29406ace0 msgr2=0x7fc29406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:52.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.570+0000 7fc2b6d9c700 1 --2- 172.21.15.43:0/1647351537 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc29406ace0 0x7fc29406d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4001360 tx=0x7fc2a4009040 comp rx=0 tx=0).stop 2023-06-28T20:07:52.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.570+0000 7fc2b6d9c700 1 -- 172.21.15.43:0/1647351537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2b015cae0 msgr2=0x7fc2b00cb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:52.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.570+0000 7fc2b6d9c700 1 --2- 172.21.15.43:0/1647351537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2b015cae0 0x7fc2b00cb210 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8014040 tx=0x7fc2a8004ad0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.570+0000 7fc2b6d9c700 1 -- 172.21.15.43:0/1647351537 shutdown_connections 2023-06-28T20:07:52.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.570+0000 7fc2b6d9c700 1 --2- 172.21.15.43:0/1647351537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2b015cae0 0x7fc2b00cb210 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.570+0000 7fc2b6d9c700 1 --2- 172.21.15.43:0/1647351537 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2b006d310 0x7fc2b01f2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.570+0000 7fc2b6d9c700 1 --2- 172.21.15.43:0/1647351537 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc29406ace0 0x7fc29406d190 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.570+0000 7fc2b6d9c700 1 --2- 172.21.15.43:0/1647351537 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc2b00cfe00 0x7fc2b015a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.570+0000 7fc2b6d9c700 1 -- 172.21.15.43:0/1647351537 >> 172.21.15.43:0/1647351537 conn(0x7fc2b00664b0 msgr2=0x7fc2b00c7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:52.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.570+0000 7fc2b6d9c700 1 -- 172.21.15.43:0/1647351537 shutdown_connections 2023-06-28T20:07:52.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.570+0000 7fc2b6d9c700 1 -- 172.21.15.43:0/1647351537 wait complete. 2023-06-28T20:07:52.585 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-28T20:07:52.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.710+0000 7f15952fd700 1 Processor -- start 2023-06-28T20:07:52.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.710+0000 7f15952fd700 1 -- start start 2023-06-28T20:07:52.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.711+0000 7f15952fd700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f159015cf10 0x7f15900b82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:52.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.711+0000 7f15952fd700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15900d0f00 con 0x7f15900d0a80 2023-06-28T20:07:52.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.711+0000 7f15952fd700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15900b8c80 con 0x7f15900cf830 2023-06-28T20:07:52.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.711+0000 7f15952fd700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15900d11a0 con 0x7f159015cf10 2023-06-28T20:07:52.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.711+0000 7f158e7fc700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f15900d0a80 0x7f15900d1c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43188/0 (socket says 172.21.15.43:43188) 2023-06-28T20:07:52.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.711+0000 7f158e7fc700 1 -- 172.21.15.43:0/2728847610 learned_addr learned my addr 172.21.15.43:0/2728847610 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:52.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.711+0000 7f158f7fe700 1 --2- 172.21.15.43:0/2728847610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f159015cf10 0x7f15900b82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:52.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.711+0000 7f158dffb700 1 -- 172.21.15.43:0/2728847610 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4007419790 0 0) 0x7f15900d0f00 con 0x7f15900d0a80 2023-06-28T20:07:52.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.711+0000 7f158dffb700 1 -- 172.21.15.43:0/2728847610 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15780035b0 con 0x7f15900d0a80 2023-06-28T20:07:52.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.712+0000 7f158dffb700 1 -- 172.21.15.43:0/2728847610 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1173310586 0 0) 0x7f15780035b0 con 0x7f15900d0a80 2023-06-28T20:07:52.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.712+0000 7f158dffb700 1 -- 172.21.15.43:0/2728847610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f159015cf10 msgr2=0x7f15900b82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:52.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.712+0000 7f158dffb700 1 --2- 172.21.15.43:0/2728847610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f159015cf10 0x7f15900b82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.712+0000 7f158dffb700 1 -- 172.21.15.43:0/2728847610 >> v1:172.21.15.178:6789/0 conn(0x7f15900cf830 legacy=0x7f15900cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:52.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.712+0000 7f158dffb700 1 -- 172.21.15.43:0/2728847610 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15900d1480 con 0x7f15900d0a80 2023-06-28T20:07:52.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.712+0000 7f158dffb700 1 -- 172.21.15.43:0/2728847610 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f1584001670 con 0x7f15900d0a80 2023-06-28T20:07:52.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.712+0000 7f158dffb700 1 -- 172.21.15.43:0/2728847610 >> v1:172.21.15.43:6789/0 conn(0x7f15900d0a80 legacy=0x7f15900d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:52.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.712+0000 7f158dffb700 1 --2- 172.21.15.43:0/2728847610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f159015cf10 0x7f1578003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:52.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.712+0000 7f158dffb700 1 --2- 172.21.15.43:0/2728847610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1578003fb0 0x7f1578004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:52.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.712+0000 7f158dffb700 1 --2- 172.21.15.43:0/2728847610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1578004d90 0x7f1578005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:52.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.712+0000 7f158dffb700 1 -- 172.21.15.43:0/2728847610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15900d11a0 con 0x7f1578004d90 2023-06-28T20:07:52.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.712+0000 7f158dffb700 1 -- 172.21.15.43:0/2728847610 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157800baf0 con 0x7f1578003fb0 2023-06-28T20:07:52.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.712+0000 7f158dffb700 1 -- 172.21.15.43:0/2728847610 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157800bcb0 con 0x7f159015cf10 2023-06-28T20:07:52.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.712+0000 7f158f7fe700 1 --2- 172.21.15.43:0/2728847610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1578004d90 0x7f1578005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:52.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.712+0000 7f158effd700 1 --2- 172.21.15.43:0/2728847610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f159015cf10 0x7f1578003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:52.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.712+0000 7f158e7fc700 1 --2- 172.21.15.43:0/2728847610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1578003fb0 0x7f1578004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:52.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.712+0000 7f158f7fe700 1 -- 172.21.15.43:0/2728847610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f159015cf10 msgr2=0x7f1578003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:52.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.712+0000 7f158f7fe700 1 --2- 172.21.15.43:0/2728847610 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f159015cf10 0x7f1578003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.712+0000 7f158f7fe700 1 -- 172.21.15.43:0/2728847610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1578003fb0 msgr2=0x7f1578004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:52.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.712+0000 7f158f7fe700 1 --2- 172.21.15.43:0/2728847610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1578003fb0 0x7f1578004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.712+0000 7f158f7fe700 1 -- 172.21.15.43:0/2728847610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15780035b0 con 0x7f1578004d90 2023-06-28T20:07:52.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.712+0000 7f158f7fe700 1 --2- 172.21.15.43:0/2728847610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1578004d90 0x7f1578005200 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f159008c160 tx=0x7f1578005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc04d4c3525f6d7 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:52.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.713+0000 7f158dffb700 1 -- 172.21.15.43:0/2728847610 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158800eba0 con 0x7f1578004d90 2023-06-28T20:07:52.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.713+0000 7f158dffb700 1 -- 172.21.15.43:0/2728847610 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158800f040 con 0x7f1578004d90 2023-06-28T20:07:52.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.713+0000 7f158dffb700 1 -- 172.21.15.43:0/2728847610 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1588014480 con 0x7f1578004d90 2023-06-28T20:07:52.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.713+0000 7f15952fd700 1 -- 172.21.15.43:0/2728847610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1578004d90 msgr2=0x7f1578005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:52.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.713+0000 7f15952fd700 1 --2- 172.21.15.43:0/2728847610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1578004d90 0x7f1578005200 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f159008c160 tx=0x7f1578005880 comp rx=0 tx=0).stop 2023-06-28T20:07:52.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.713+0000 7f158f7fe700 1 -- 172.21.15.43:0/2728847610 reap_dead start 2023-06-28T20:07:52.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.713+0000 7f15952fd700 1 -- 172.21.15.43:0/2728847610 shutdown_connections 2023-06-28T20:07:52.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.713+0000 7f15952fd700 1 -- 172.21.15.43:0/2728847610 >> 172.21.15.43:0/2728847610 conn(0x7f159006a640 msgr2=0x7f159006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:52.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.713+0000 7f15952fd700 1 -- 172.21.15.43:0/2728847610 shutdown_connections 2023-06-28T20:07:52.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.713+0000 7f15952fd700 1 -- 172.21.15.43:0/2728847610 wait complete. 2023-06-28T20:07:52.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.715+0000 7f15952fd700 1 Processor -- start 2023-06-28T20:07:52.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.715+0000 7f15952fd700 1 -- start start 2023-06-28T20:07:52.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.715+0000 7f15952fd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15901f5770 0x7f15900b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:52.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.715+0000 7f15952fd700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f15901f2650 0x7f15901f3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:52.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.715+0000 7f15952fd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15901f3db0 0x7f15900cc040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:52.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.715+0000 7f15952fd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15901f3710 con 0x7f15901f3db0 2023-06-28T20:07:52.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.715+0000 7f15952fd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1590157cc0 con 0x7f15901f5770 2023-06-28T20:07:52.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.715+0000 7f15952fd700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15901f2e70 con 0x7f15901f2650 2023-06-28T20:07:52.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.716+0000 7f158f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15901f3db0 0x7f15900cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:52.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.716+0000 7f158f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15901f3db0 0x7f15900cc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54016/0 (socket says 172.21.15.43:54016) 2023-06-28T20:07:52.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.716+0000 7f158f7fe700 1 -- 172.21.15.43:0/3077271250 learned_addr learned my addr 172.21.15.43:0/3077271250 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:52.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.716+0000 7f158e7fc700 1 --2- 172.21.15.43:0/3077271250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f15901f2650 0x7f15901f3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:52.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.716+0000 7f158effd700 1 --2- 172.21.15.43:0/3077271250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15901f5770 0x7f15900b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:52.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.716+0000 7f158f7fe700 1 -- 172.21.15.43:0/3077271250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f15901f2650 msgr2=0x7f15901f3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:52.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.716+0000 7f158f7fe700 1 --2- 172.21.15.43:0/3077271250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f15901f2650 0x7f15901f3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.716+0000 7f158f7fe700 1 -- 172.21.15.43:0/3077271250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15901f5770 msgr2=0x7f15900b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:52.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.716+0000 7f158f7fe700 1 --2- 172.21.15.43:0/3077271250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15901f5770 0x7f15900b4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.716+0000 7f158f7fe700 1 -- 172.21.15.43:0/3077271250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15901f3150 con 0x7f15901f3db0 2023-06-28T20:07:52.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.716+0000 7f158f7fe700 1 --2- 172.21.15.43:0/3077271250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15901f3db0 0x7f15900cc040 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f157800bac0 tx=0x7f1588004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:52.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.716+0000 7f15777fe700 1 -- 172.21.15.43:0/3077271250 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1588007000 con 0x7f15901f3db0 2023-06-28T20:07:52.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.716+0000 7f15777fe700 1 -- 172.21.15.43:0/3077271250 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1588022070 con 0x7f15901f3db0 2023-06-28T20:07:52.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.716+0000 7f15777fe700 1 -- 172.21.15.43:0/3077271250 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1588003c20 con 0x7f15901f3db0 2023-06-28T20:07:52.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.716+0000 7f15952fd700 1 -- 172.21.15.43:0/3077271250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15901f3430 con 0x7f15901f3db0 2023-06-28T20:07:52.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.716+0000 7f15952fd700 1 -- 172.21.15.43:0/3077271250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15900cc580 con 0x7f15901f3db0 2023-06-28T20:07:52.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.717+0000 7f15952fd700 1 -- 172.21.15.43:0/3077271250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1590157e70 con 0x7f15901f3db0 2023-06-28T20:07:52.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.718+0000 7f15777fe700 1 -- 172.21.15.43:0/3077271250 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f15880159d0 con 0x7f15901f3db0 2023-06-28T20:07:52.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.718+0000 7f15777fe700 1 --2- 172.21.15.43:0/3077271250 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f157c06ace0 0x7f157c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:52.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.718+0000 7f15777fe700 1 -- 172.21.15.43:0/3077271250 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1588024070 con 0x7f15901f3db0 2023-06-28T20:07:52.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.718+0000 7f158effd700 1 --2- 172.21.15.43:0/3077271250 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f157c06ace0 0x7f157c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:52.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.718+0000 7f158effd700 1 --2- 172.21.15.43:0/3077271250 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f157c06ace0 0x7f157c06d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f15900d1dd0 tx=0x7f1580009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:52.737 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.722+0000 7f15777fe700 1 -- 172.21.15.43:0/3077271250 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1590157e70 con 0x7f15901f3db0 2023-06-28T20:07:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.921+0000 7f15952fd700 1 -- 172.21.15.43:0/3077271250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1590157e30 con 0x7f15901f3db0 2023-06-28T20:07:52.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.922+0000 7f15777fe700 1 -- 172.21.15.43:0/3077271250 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f158802d020 con 0x7f15901f3db0 2023-06-28T20:07:52.925 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-06-28T20:07:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.924+0000 7f15952fd700 1 -- 172.21.15.43:0/3077271250 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f157c06ace0 msgr2=0x7f157c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.924+0000 7f15952fd700 1 --2- 172.21.15.43:0/3077271250 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f157c06ace0 0x7f157c06d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f15900d1dd0 tx=0x7f1580009040 comp rx=0 tx=0).stop 2023-06-28T20:07:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.924+0000 7f15952fd700 1 -- 172.21.15.43:0/3077271250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15901f3db0 msgr2=0x7f15900cc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.924+0000 7f15952fd700 1 --2- 172.21.15.43:0/3077271250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15901f3db0 0x7f15900cc040 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f157800bac0 tx=0x7f1588004b40 comp rx=0 tx=0).stop 2023-06-28T20:07:52.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.924+0000 7f15952fd700 1 -- 172.21.15.43:0/3077271250 shutdown_connections 2023-06-28T20:07:52.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.924+0000 7f15952fd700 1 --2- 172.21.15.43:0/3077271250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15901f3db0 0x7f15900cc040 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.924+0000 7f15952fd700 1 --2- 172.21.15.43:0/3077271250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f15901f5770 0x7f15900b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.924+0000 7f15952fd700 1 --2- 172.21.15.43:0/3077271250 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f157c06ace0 0x7f157c06d190 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.924+0000 7f15952fd700 1 --2- 172.21.15.43:0/3077271250 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f15901f2650 0x7f15901f3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.924+0000 7f15952fd700 1 -- 172.21.15.43:0/3077271250 >> 172.21.15.43:0/3077271250 conn(0x7f159006a640 msgr2=0x7f159006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.924+0000 7f15952fd700 1 -- 172.21.15.43:0/3077271250 shutdown_connections 2023-06-28T20:07:52.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:52.924+0000 7f15952fd700 1 -- 172.21.15.43:0/3077271250 wait complete. 2023-06-28T20:07:52.959 INFO:tasks.cephfs_test_runner:Starting test: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-06-28T20:07:52.961 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values' 2023-06-28T20:07:53.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.087+0000 7f8f2edd5700 1 Processor -- start 2023-06-28T20:07:53.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.087+0000 7f8f2edd5700 1 -- start start 2023-06-28T20:07:53.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.088+0000 7f8f2edd5700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8f2815d9f0 0x7f8f2806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:53.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.088+0000 7f8f2edd5700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8f28070cb0 0x7f8f28069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:53.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.088+0000 7f8f2edd5700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f2806a850 con 0x7f8f2806a430 2023-06-28T20:07:53.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.088+0000 7f8f2edd5700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f2815ddc0 con 0x7f8f2815d9f0 2023-06-28T20:07:53.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.088+0000 7f8f2edd5700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f2806f3e0 con 0x7f8f28070cb0 2023-06-28T20:07:53.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.088+0000 7f8f2cb71700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8f2815d9f0 0x7f8f2806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:53.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.088+0000 7f8f2cb71700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8f2815d9f0 0x7f8f2806d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54020/0 (socket says 172.21.15.43:54020) 2023-06-28T20:07:53.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.088+0000 7f8f2cb71700 1 -- 172.21.15.43:0/418729702 learned_addr learned my addr 172.21.15.43:0/418729702 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:53.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.089+0000 7f8f2cb71700 1 -- 172.21.15.43:0/418729702 >> v2:172.21.15.178:3300/0 conn(0x7f8f28070cb0 msgr2=0x7f8f28069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:53.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.089+0000 7f8f2cb71700 1 --2- 172.21.15.43:0/418729702 >> v2:172.21.15.178:3300/0 conn(0x7f8f28070cb0 0x7f8f28069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:53.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.089+0000 7f8f2cb71700 1 -- 172.21.15.43:0/418729702 >> v1:172.21.15.178:6789/0 conn(0x7f8f2806a430 legacy=0x7f8f2815df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:53.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.089+0000 7f8f2cb71700 1 -- 172.21.15.43:0/418729702 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f28071170 con 0x7f8f2815d9f0 2023-06-28T20:07:53.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.089+0000 7f8f2cb71700 1 --2- 172.21.15.43:0/418729702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f2815d9f0 0x7f8f2806d3b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8f2806aee0 tx=0x7f8f1c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04eb4bb7d3f96dc server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:53.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.089+0000 7f8f277fe700 1 -- 172.21.15.43:0/418729702 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c004230 con 0x7f8f2815d9f0 2023-06-28T20:07:53.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.089+0000 7f8f277fe700 1 -- 172.21.15.43:0/418729702 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f1c004390 con 0x7f8f2815d9f0 2023-06-28T20:07:53.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.089+0000 7f8f277fe700 1 -- 172.21.15.43:0/418729702 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c0036a0 con 0x7f8f2815d9f0 2023-06-28T20:07:53.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.090+0000 7f8f2edd5700 1 -- 172.21.15.43:0/418729702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f2815d9f0 msgr2=0x7f8f2806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:53.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.090+0000 7f8f2edd5700 1 --2- 172.21.15.43:0/418729702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f2815d9f0 0x7f8f2806d3b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8f2806aee0 tx=0x7f8f1c009c00 comp rx=0 tx=0).stop 2023-06-28T20:07:53.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.090+0000 7f8f2edd5700 1 -- 172.21.15.43:0/418729702 shutdown_connections 2023-06-28T20:07:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.090+0000 7f8f2edd5700 1 --2- 172.21.15.43:0/418729702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f2815d9f0 0x7f8f2806d3b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.090+0000 7f8f2edd5700 1 --2- 172.21.15.43:0/418729702 >> v2:172.21.15.178:3300/0 conn(0x7f8f28070cb0 0x7f8f28069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.090+0000 7f8f2edd5700 1 -- 172.21.15.43:0/418729702 >> 172.21.15.43:0/418729702 conn(0x7f8f280b1720 msgr2=0x7f8f280b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.090+0000 7f8f2edd5700 1 -- 172.21.15.43:0/418729702 shutdown_connections 2023-06-28T20:07:53.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.090+0000 7f8f2edd5700 1 -- 172.21.15.43:0/418729702 wait complete. 2023-06-28T20:07:53.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.091+0000 7f8f2edd5700 1 Processor -- start 2023-06-28T20:07:53.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.091+0000 7f8f2edd5700 1 -- start start 2023-06-28T20:07:53.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.092+0000 7f8f2edd5700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f2806a430 0x7f8f28153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:53.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.092+0000 7f8f2edd5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f28070cb0 0x7f8f28145c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:53.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.092+0000 7f8f2edd5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f2815d9f0 0x7f8f281461b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:53.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.092+0000 7f8f2edd5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f28153530 con 0x7f8f28070cb0 2023-06-28T20:07:53.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.092+0000 7f8f2edd5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f280c9350 con 0x7f8f2815d9f0 2023-06-28T20:07:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.092+0000 7f8f2edd5700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f280c9630 con 0x7f8f2806a430 2023-06-28T20:07:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.092+0000 7f8f27fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f28070cb0 0x7f8f28145c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.092+0000 7f8f27fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f28070cb0 0x7f8f28145c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54028/0 (socket says 172.21.15.43:54028) 2023-06-28T20:07:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.092+0000 7f8f27fff700 1 -- 172.21.15.43:0/1795912721 learned_addr learned my addr 172.21.15.43:0/1795912721 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.092+0000 7f8f2cb71700 1 --2- 172.21.15.43:0/1795912721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f2806a430 0x7f8f28153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.092+0000 7f8f2d372700 1 --2- 172.21.15.43:0/1795912721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f2815d9f0 0x7f8f281461b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.092+0000 7f8f27fff700 1 -- 172.21.15.43:0/1795912721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f2806a430 msgr2=0x7f8f28153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.092+0000 7f8f27fff700 1 --2- 172.21.15.43:0/1795912721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f2806a430 0x7f8f28153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.092+0000 7f8f27fff700 1 -- 172.21.15.43:0/1795912721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f2815d9f0 msgr2=0x7f8f281461b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.092+0000 7f8f27fff700 1 --2- 172.21.15.43:0/1795912721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f2815d9f0 0x7f8f281461b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.092+0000 7f8f27fff700 1 -- 172.21.15.43:0/1795912721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1c009550 con 0x7f8f28070cb0 2023-06-28T20:07:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.092+0000 7f8f27fff700 1 --2- 172.21.15.43:0/1795912721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f28070cb0 0x7f8f28145c70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400b500 tx=0x7f8f1400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.092+0000 7f8f257fa700 1 -- 172.21.15.43:0/1795912721 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f14011830 con 0x7f8f28070cb0 2023-06-28T20:07:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.093+0000 7f8f257fa700 1 -- 172.21.15.43:0/1795912721 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f14011990 con 0x7f8f28070cb0 2023-06-28T20:07:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.093+0000 7f8f2edd5700 1 -- 172.21.15.43:0/1795912721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f2815a610 con 0x7f8f28070cb0 2023-06-28T20:07:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.093+0000 7f8f257fa700 1 -- 172.21.15.43:0/1795912721 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f14011d40 con 0x7f8f28070cb0 2023-06-28T20:07:53.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.093+0000 7f8f2edd5700 1 -- 172.21.15.43:0/1795912721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f2815abd0 con 0x7f8f28070cb0 2023-06-28T20:07:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.093+0000 7f8f2edd5700 1 -- 172.21.15.43:0/1795912721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f280728b0 con 0x7f8f28070cb0 2023-06-28T20:07:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.094+0000 7f8f257fa700 1 -- 172.21.15.43:0/1795912721 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8f1401a400 con 0x7f8f28070cb0 2023-06-28T20:07:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.094+0000 7f8f257fa700 1 --2- 172.21.15.43:0/1795912721 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f8f0c06ace0 0x7f8f0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:53.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.095+0000 7f8f257fa700 1 -- 172.21.15.43:0/1795912721 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8f14089500 con 0x7f8f28070cb0 2023-06-28T20:07:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.095+0000 7f8f2cb71700 1 --2- 172.21.15.43:0/1795912721 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f8f0c06ace0 0x7f8f0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:53.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.095+0000 7f8f2cb71700 1 --2- 172.21.15.43:0/1795912721 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f8f0c06ace0 0x7f8f0c06d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009870 tx=0x7f8f1c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:53.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.099+0000 7f8f257fa700 1 -- 172.21.15.43:0/1795912721 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8f140545b0 con 0x7f8f28070cb0 2023-06-28T20:07:53.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:53.382+0000 7f8f2edd5700 1 -- 172.21.15.43:0/1795912721 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]} v 0) v1 -- 0x7f8f2806e4c0 con 0x7f8f28070cb0 2023-06-28T20:07:54.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.070+0000 7f8f257fa700 1 -- 172.21.15.43:0/1795912721 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]}]=0 v48) v1 ==== 163+0+0 (secure 0 0 0) 0x7f8f14057bd0 con 0x7f8f28070cb0 2023-06-28T20:07:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.073+0000 7f8f2edd5700 1 -- 172.21.15.43:0/1795912721 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f8f0c06ace0 msgr2=0x7f8f0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.073+0000 7f8f2edd5700 1 --2- 172.21.15.43:0/1795912721 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f8f0c06ace0 0x7f8f0c06d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009870 tx=0x7f8f1c008030 comp rx=0 tx=0).stop 2023-06-28T20:07:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.073+0000 7f8f2edd5700 1 -- 172.21.15.43:0/1795912721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f28070cb0 msgr2=0x7f8f28145c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.073+0000 7f8f2edd5700 1 --2- 172.21.15.43:0/1795912721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f28070cb0 0x7f8f28145c70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8f1400b500 tx=0x7f8f1400b8c0 comp rx=0 tx=0).stop 2023-06-28T20:07:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.073+0000 7f8f2edd5700 1 -- 172.21.15.43:0/1795912721 shutdown_connections 2023-06-28T20:07:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.073+0000 7f8f2edd5700 1 --2- 172.21.15.43:0/1795912721 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f2815d9f0 0x7f8f281461b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:54.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.074+0000 7f8f2edd5700 1 --2- 172.21.15.43:0/1795912721 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f8f0c06ace0 0x7f8f0c06d190 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:54.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.074+0000 7f8f2edd5700 1 --2- 172.21.15.43:0/1795912721 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8f2806a430 0x7f8f28153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:54.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.074+0000 7f8f2edd5700 1 --2- 172.21.15.43:0/1795912721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f28070cb0 0x7f8f28145c70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:54.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.074+0000 7f8f2edd5700 1 -- 172.21.15.43:0/1795912721 >> 172.21.15.43:0/1795912721 conn(0x7f8f280b1720 msgr2=0x7f8f280b0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:54.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.074+0000 7f8f2edd5700 1 -- 172.21.15.43:0/1795912721 shutdown_connections 2023-06-28T20:07:54.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.074+0000 7f8f2edd5700 1 -- 172.21.15.43:0/1795912721 wait complete. 2023-06-28T20:07:54.092 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-06-28T20:07:54.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.224+0000 7f2f810bb700 1 Processor -- start 2023-06-28T20:07:54.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.225+0000 7f2f810bb700 1 -- start start 2023-06-28T20:07:54.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.225+0000 7f2f810bb700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2f7c159e80 0x7f2f7c15a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:54.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.225+0000 7f2f810bb700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f7c1425a0 0x7f2f7c0cce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:54.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.225+0000 7f2f810bb700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f7c06a090 con 0x7f2f7c0c9900 2023-06-28T20:07:54.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.225+0000 7f2f810bb700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f7c15a940 con 0x7f2f7c159e80 2023-06-28T20:07:54.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.225+0000 7f2f810bb700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c06a370 con 0x7f2f7c1425a0 2023-06-28T20:07:54.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.226+0000 7f2f7ad9d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2f7c159e80 0x7f2f7c15a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:54.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.226+0000 7f2f7b59e700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2f7c0c9900 0x7f2f7c06ae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43224/0 (socket says 172.21.15.43:43224) 2023-06-28T20:07:54.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.226+0000 7f2f7b59e700 1 -- 172.21.15.43:0/2596022412 learned_addr learned my addr 172.21.15.43:0/2596022412 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:54.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.226+0000 7f2f7a59c700 1 --2- 172.21.15.43:0/2596022412 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f7c1425a0 0x7f2f7c0cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:54.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.226+0000 7f2f7ad9d700 1 -- 172.21.15.43:0/2596022412 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f7c1425a0 msgr2=0x7f2f7c0cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:54.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.226+0000 7f2f7ad9d700 1 --2- 172.21.15.43:0/2596022412 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f7c1425a0 0x7f2f7c0cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:54.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.226+0000 7f2f7ad9d700 1 -- 172.21.15.43:0/2596022412 >> v1:172.21.15.43:6789/0 conn(0x7f2f7c0c9900 legacy=0x7f2f7c06ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:54.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.226+0000 7f2f79d9b700 1 -- 172.21.15.43:0/2596022412 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2343653613 0 0) 0x7f2f7c06a090 con 0x7f2f7c0c9900 2023-06-28T20:07:54.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.226+0000 7f2f7ad9d700 1 -- 172.21.15.43:0/2596022412 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f7c06a650 con 0x7f2f7c159e80 2023-06-28T20:07:54.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.226+0000 7f2f7a59c700 1 --2- 172.21.15.43:0/2596022412 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f7c1425a0 0x7f2f7c0cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:54.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.226+0000 7f2f7ad9d700 1 --2- 172.21.15.43:0/2596022412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f7c159e80 0x7f2f7c15a250 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c009840 tx=0x7f2f6c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1c49d663f98ecc server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:54.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.226+0000 7f2f79d9b700 1 -- 172.21.15.43:0/2596022412 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c011830 con 0x7f2f7c159e80 2023-06-28T20:07:54.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.226+0000 7f2f79d9b700 1 -- 172.21.15.43:0/2596022412 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6c011990 con 0x7f2f7c159e80 2023-06-28T20:07:54.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.226+0000 7f2f79d9b700 1 -- 172.21.15.43:0/2596022412 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c011d20 con 0x7f2f7c159e80 2023-06-28T20:07:54.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.227+0000 7f2f810bb700 1 -- 172.21.15.43:0/2596022412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f7c159e80 msgr2=0x7f2f7c15a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:54.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.227+0000 7f2f810bb700 1 --2- 172.21.15.43:0/2596022412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f7c159e80 0x7f2f7c15a250 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c009840 tx=0x7f2f6c009b50 comp rx=0 tx=0).stop 2023-06-28T20:07:54.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.227+0000 7f2f810bb700 1 -- 172.21.15.43:0/2596022412 shutdown_connections 2023-06-28T20:07:54.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.227+0000 7f2f810bb700 1 --2- 172.21.15.43:0/2596022412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f7c159e80 0x7f2f7c15a250 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:54.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.227+0000 7f2f810bb700 1 --2- 172.21.15.43:0/2596022412 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f7c1425a0 0x7f2f7c0cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:54.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.227+0000 7f2f810bb700 1 -- 172.21.15.43:0/2596022412 >> 172.21.15.43:0/2596022412 conn(0x7f2f7c0b1a50 msgr2=0x7f2f7c0cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:54.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.227+0000 7f2f810bb700 1 -- 172.21.15.43:0/2596022412 shutdown_connections 2023-06-28T20:07:54.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.227+0000 7f2f810bb700 1 -- 172.21.15.43:0/2596022412 wait complete. 2023-06-28T20:07:54.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.228+0000 7f2f810bb700 1 Processor -- start 2023-06-28T20:07:54.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.228+0000 7f2f810bb700 1 -- start start 2023-06-28T20:07:54.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.229+0000 7f2f810bb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f7c0c9900 0x7f2f7c1583a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:54.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.229+0000 7f2f810bb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f7c1425a0 0x7f2f7c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:54.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.229+0000 7f2f810bb700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f7c159e80 0x7f2f7c0c30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:54.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.229+0000 7f2f810bb700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c158ea0 con 0x7f2f7c1425a0 2023-06-28T20:07:54.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.229+0000 7f2f810bb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c0b3290 con 0x7f2f7c0c9900 2023-06-28T20:07:54.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.229+0000 7f2f810bb700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c0b3570 con 0x7f2f7c159e80 2023-06-28T20:07:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.229+0000 7f2f7a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f7c1425a0 0x7f2f7c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.229+0000 7f2f7a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f7c1425a0 0x7f2f7c15aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54042/0 (socket says 172.21.15.43:54042) 2023-06-28T20:07:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.229+0000 7f2f7a59c700 1 -- 172.21.15.43:0/1145847133 learned_addr learned my addr 172.21.15.43:0/1145847133 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.230+0000 7f2f7b59e700 1 --2- 172.21.15.43:0/1145847133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f7c159e80 0x7f2f7c0c30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.230+0000 7f2f7a59c700 1 -- 172.21.15.43:0/1145847133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f7c159e80 msgr2=0x7f2f7c0c30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.230+0000 7f2f7a59c700 1 --2- 172.21.15.43:0/1145847133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f7c159e80 0x7f2f7c0c30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.230+0000 7f2f7ad9d700 1 --2- 172.21.15.43:0/1145847133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f7c0c9900 0x7f2f7c1583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:54.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.230+0000 7f2f7a59c700 1 -- 172.21.15.43:0/1145847133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f7c0c9900 msgr2=0x7f2f7c1583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:54.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.230+0000 7f2f7a59c700 1 --2- 172.21.15.43:0/1145847133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f7c0c9900 0x7f2f7c1583a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:54.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.230+0000 7f2f7a59c700 1 -- 172.21.15.43:0/1145847133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6c0094a0 con 0x7f2f7c1425a0 2023-06-28T20:07:54.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.230+0000 7f2f7a59c700 1 --2- 172.21.15.43:0/1145847133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f7c1425a0 0x7f2f7c15aa80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400c7f0 tx=0x7f2f6400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:54.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.230+0000 7f2f737fe700 1 -- 172.21.15.43:0/1145847133 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f64011800 con 0x7f2f7c1425a0 2023-06-28T20:07:54.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.230+0000 7f2f737fe700 1 -- 172.21.15.43:0/1145847133 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f64011960 con 0x7f2f7c1425a0 2023-06-28T20:07:54.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.230+0000 7f2f737fe700 1 -- 172.21.15.43:0/1145847133 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f64011d10 con 0x7f2f7c1425a0 2023-06-28T20:07:54.238 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.230+0000 7f2f810bb700 1 -- 172.21.15.43:0/1145847133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f7c0b3850 con 0x7f2f7c1425a0 2023-06-28T20:07:54.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.230+0000 7f2f810bb700 1 -- 172.21.15.43:0/1145847133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7c0b3e10 con 0x7f2f7c1425a0 2023-06-28T20:07:54.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.231+0000 7f2f810bb700 1 -- 172.21.15.43:0/1145847133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f7c0728b0 con 0x7f2f7c1425a0 2023-06-28T20:07:54.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.231+0000 7f2f737fe700 1 -- 172.21.15.43:0/1145847133 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2f64018360 con 0x7f2f7c1425a0 2023-06-28T20:07:54.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.232+0000 7f2f737fe700 1 --2- 172.21.15.43:0/1145847133 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2f5c06ace0 0x7f2f5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:54.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.232+0000 7f2f737fe700 1 -- 172.21.15.43:0/1145847133 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2f64089500 con 0x7f2f7c1425a0 2023-06-28T20:07:54.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.232+0000 7f2f7ad9d700 1 --2- 172.21.15.43:0/1145847133 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2f5c06ace0 0x7f2f5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:54.239 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.232+0000 7f2f7ad9d700 1 --2- 172.21.15.43:0/1145847133 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2f5c06ace0 0x7f2f5c06d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c000c00 tx=0x7f2f6c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:54.240 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.236+0000 7f2f737fe700 1 -- 172.21.15.43:0/1145847133 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2f64054530 con 0x7f2f7c1425a0 2023-06-28T20:07:54.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.435+0000 7f2f810bb700 1 -- 172.21.15.43:0/1145847133 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2f7c159140 con 0x7f2f7c1425a0 2023-06-28T20:07:54.441 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.438+0000 7f2f737fe700 1 -- 172.21.15.43:0/1145847133 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2f64057b50 con 0x7f2f7c1425a0 2023-06-28T20:07:54.443 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.440+0000 7f2f810bb700 1 -- 172.21.15.43:0/1145847133 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2f5c06ace0 msgr2=0x7f2f5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:54.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.440+0000 7f2f810bb700 1 --2- 172.21.15.43:0/1145847133 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2f5c06ace0 0x7f2f5c06d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c000c00 tx=0x7f2f6c008030 comp rx=0 tx=0).stop 2023-06-28T20:07:54.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.440+0000 7f2f810bb700 1 -- 172.21.15.43:0/1145847133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f7c1425a0 msgr2=0x7f2f7c15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:54.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.440+0000 7f2f810bb700 1 --2- 172.21.15.43:0/1145847133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f7c1425a0 0x7f2f7c15aa80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2f6400c7f0 tx=0x7f2f6400cbb0 comp rx=0 tx=0).stop 2023-06-28T20:07:54.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.440+0000 7f2f810bb700 1 -- 172.21.15.43:0/1145847133 shutdown_connections 2023-06-28T20:07:54.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.440+0000 7f2f810bb700 1 --2- 172.21.15.43:0/1145847133 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2f5c06ace0 0x7f2f5c06d190 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:54.444 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.440+0000 7f2f810bb700 1 --2- 172.21.15.43:0/1145847133 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f7c159e80 0x7f2f7c0c30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:54.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.440+0000 7f2f810bb700 1 --2- 172.21.15.43:0/1145847133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f7c0c9900 0x7f2f7c1583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:54.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.440+0000 7f2f810bb700 1 --2- 172.21.15.43:0/1145847133 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f7c1425a0 0x7f2f7c15aa80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:54.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.440+0000 7f2f810bb700 1 -- 172.21.15.43:0/1145847133 >> 172.21.15.43:0/1145847133 conn(0x7f2f7c0b1a50 msgr2=0x7f2f7c0cf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:54.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.440+0000 7f2f810bb700 1 -- 172.21.15.43:0/1145847133 shutdown_connections 2023-06-28T20:07:54.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.440+0000 7f2f810bb700 1 -- 172.21.15.43:0/1145847133 wait complete. 2023-06-28T20:07:54.456 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-06-28T20:07:54.456 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-06-28T20:07:54.498 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-06-28T20:07:54.547 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-28T20:07:54.549 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-28T20:07:54.551 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-06-28T20:07:54.551 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:07:54.551 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:07:54.608 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:07:54.608 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:07:54.665 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:07:54.665 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:07:54.734 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:07:54.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.896+0000 7f402cbac700 1 Processor -- start 2023-06-28T20:07:54.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.896+0000 7f402cbac700 1 -- start start 2023-06-28T20:07:54.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.897+0000 7f402cbac700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f40280d0630 0x7f40280d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:54.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.897+0000 7f402cbac700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f402806b710 0x7f40280ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:54.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.897+0000 7f402cbac700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40280ccf10 con 0x7f40280cbd40 2023-06-28T20:07:54.901 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.897+0000 7f402cbac700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f402815cd80 con 0x7f40280d0630 2023-06-28T20:07:54.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.897+0000 7f402cbac700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40280d0a00 con 0x7f402806b710 2023-06-28T20:07:54.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.897+0000 7f402659c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f40280d0630 0x7f40280d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:54.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.897+0000 7f4025d9b700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f40280cbd40 0x7f402806b000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43234/0 (socket says 172.21.15.43:43234) 2023-06-28T20:07:54.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.897+0000 7f4025d9b700 1 -- 172.21.15.43:0/4150379448 learned_addr learned my addr 172.21.15.43:0/4150379448 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:54.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.898+0000 7f4026d9d700 1 --2- 172.21.15.43:0/4150379448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f402806b710 0x7f40280ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:54.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.898+0000 7f402659c700 1 -- 172.21.15.43:0/4150379448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f402806b710 msgr2=0x7f40280ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:54.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.898+0000 7f402659c700 1 --2- 172.21.15.43:0/4150379448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f402806b710 0x7f40280ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:54.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.898+0000 7f402659c700 1 -- 172.21.15.43:0/4150379448 >> v1:172.21.15.43:6789/0 conn(0x7f40280cbd40 legacy=0x7f402806b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:54.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.898+0000 7f402559a700 1 -- 172.21.15.43:0/4150379448 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4228980978 0 0) 0x7f40280ccf10 con 0x7f40280cbd40 2023-06-28T20:07:54.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.898+0000 7f402659c700 1 -- 172.21.15.43:0/4150379448 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40280cc300 con 0x7f40280d0630 2023-06-28T20:07:54.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.898+0000 7f4026d9d700 1 --2- 172.21.15.43:0/4150379448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f402806b710 0x7f40280ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:54.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.898+0000 7f402659c700 1 --2- 172.21.15.43:0/4150379448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40280d0630 0x7f40280d1070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4018009850 tx=0x7f4018009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d34b6de6ad3249d server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:54.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.898+0000 7f402559a700 1 -- 172.21.15.43:0/4150379448 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018005e30 con 0x7f40280d0630 2023-06-28T20:07:54.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.898+0000 7f402559a700 1 -- 172.21.15.43:0/4150379448 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4018006030 con 0x7f40280d0630 2023-06-28T20:07:54.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.898+0000 7f402559a700 1 -- 172.21.15.43:0/4150379448 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40180063c0 con 0x7f40280d0630 2023-06-28T20:07:54.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.899+0000 7f402cbac700 1 -- 172.21.15.43:0/4150379448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40280d0630 msgr2=0x7f40280d1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:54.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.899+0000 7f402cbac700 1 --2- 172.21.15.43:0/4150379448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40280d0630 0x7f40280d1070 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4018009850 tx=0x7f4018009b60 comp rx=0 tx=0).stop 2023-06-28T20:07:54.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.899+0000 7f402cbac700 1 -- 172.21.15.43:0/4150379448 shutdown_connections 2023-06-28T20:07:54.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.899+0000 7f402cbac700 1 --2- 172.21.15.43:0/4150379448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f402806b710 0x7f40280ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:54.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.899+0000 7f402cbac700 1 --2- 172.21.15.43:0/4150379448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f40280d0630 0x7f40280d1070 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:54.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.899+0000 7f402cbac700 1 -- 172.21.15.43:0/4150379448 >> 172.21.15.43:0/4150379448 conn(0x7f40280664b0 msgr2=0x7f402809a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:54.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.899+0000 7f402cbac700 1 -- 172.21.15.43:0/4150379448 shutdown_connections 2023-06-28T20:07:54.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.899+0000 7f402cbac700 1 -- 172.21.15.43:0/4150379448 wait complete. 2023-06-28T20:07:54.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.900+0000 7f402cbac700 1 Processor -- start 2023-06-28T20:07:54.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.900+0000 7f402cbac700 1 -- start start 2023-06-28T20:07:54.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.901+0000 7f402cbac700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f402806b710 0x7f40280c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:54.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.901+0000 7f402cbac700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40280cbd40 0x7f4028157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:54.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.901+0000 7f402cbac700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40280d0630 0x7f4028157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:54.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.901+0000 7f402cbac700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40281f1c50 con 0x7f402806b710 2023-06-28T20:07:54.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.901+0000 7f402cbac700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40280ca0f0 con 0x7f40280d0630 2023-06-28T20:07:54.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.901+0000 7f402cbac700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40280ca3d0 con 0x7f40280cbd40 2023-06-28T20:07:54.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.901+0000 7f402659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f402806b710 0x7f40280c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:54.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.901+0000 7f402659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f402806b710 0x7f40280c9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54054/0 (socket says 172.21.15.43:54054) 2023-06-28T20:07:54.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.901+0000 7f402659c700 1 -- 172.21.15.43:0/4183296002 learned_addr learned my addr 172.21.15.43:0/4183296002 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:54.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.901+0000 7f4026d9d700 1 --2- 172.21.15.43:0/4183296002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40280d0630 0x7f4028157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:54.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.901+0000 7f4025d9b700 1 --2- 172.21.15.43:0/4183296002 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40280cbd40 0x7f4028157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:54.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.901+0000 7f402659c700 1 -- 172.21.15.43:0/4183296002 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40280cbd40 msgr2=0x7f4028157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:54.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.901+0000 7f402659c700 1 --2- 172.21.15.43:0/4183296002 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40280cbd40 0x7f4028157860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:54.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.901+0000 7f402659c700 1 -- 172.21.15.43:0/4183296002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40280d0630 msgr2=0x7f4028157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:54.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.901+0000 7f402659c700 1 --2- 172.21.15.43:0/4183296002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40280d0630 0x7f4028157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:54.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.901+0000 7f402659c700 1 -- 172.21.15.43:0/4183296002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4018009430 con 0x7f402806b710 2023-06-28T20:07:54.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.902+0000 7f402659c700 1 --2- 172.21.15.43:0/4183296002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f402806b710 0x7f40280c9810 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4018009ba0 tx=0x7f4018004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:54.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.902+0000 7f4016ffd700 1 -- 172.21.15.43:0/4183296002 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018004450 con 0x7f402806b710 2023-06-28T20:07:54.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.902+0000 7f4016ffd700 1 -- 172.21.15.43:0/4183296002 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40180045b0 con 0x7f402806b710 2023-06-28T20:07:54.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.902+0000 7f4016ffd700 1 -- 172.21.15.43:0/4183296002 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40180038c0 con 0x7f402806b710 2023-06-28T20:07:54.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.902+0000 7f402cbac700 1 -- 172.21.15.43:0/4183296002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40280b3cd0 con 0x7f402806b710 2023-06-28T20:07:54.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.902+0000 7f402cbac700 1 -- 172.21.15.43:0/4183296002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40280ca6b0 con 0x7f402806b710 2023-06-28T20:07:54.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.903+0000 7f402cbac700 1 -- 172.21.15.43:0/4183296002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40280728b0 con 0x7f402806b710 2023-06-28T20:07:54.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.903+0000 7f4016ffd700 1 -- 172.21.15.43:0/4183296002 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4018022070 con 0x7f402806b710 2023-06-28T20:07:54.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.903+0000 7f4016ffd700 1 --2- 172.21.15.43:0/4183296002 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f400806ace0 0x7f400806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:54.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.904+0000 7f4016ffd700 1 -- 172.21.15.43:0/4183296002 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f401808ae10 con 0x7f402806b710 2023-06-28T20:07:54.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.904+0000 7f4025d9b700 1 --2- 172.21.15.43:0/4183296002 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f400806ace0 0x7f400806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:54.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.904+0000 7f4025d9b700 1 --2- 172.21.15.43:0/4183296002 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f400806ace0 0x7f400806d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4028149d40 tx=0x7f401c007be0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:54.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:54.908+0000 7f4016ffd700 1 -- 172.21.15.43:0/4183296002 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4018055f50 con 0x7f402806b710 2023-06-28T20:07:55.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.109+0000 7f402cbac700 1 -- 172.21.15.43:0/4183296002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f402806e600 con 0x7f402806b710 2023-06-28T20:07:55.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.110+0000 7f4016ffd700 1 -- 172.21.15.43:0/4183296002 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18221 (secure 0 0 0) 0x7f401802a020 con 0x7f402806b710 2023-06-28T20:07:55.113 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:55.113 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:07:22.377248+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:07:18.624759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-28T20:07:19.760017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:07:55.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.112+0000 7f402cbac700 1 -- 172.21.15.43:0/4183296002 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f400806ace0 msgr2=0x7f400806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:55.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.112+0000 7f402cbac700 1 --2- 172.21.15.43:0/4183296002 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f400806ace0 0x7f400806d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4028149d40 tx=0x7f401c007be0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.112+0000 7f402cbac700 1 -- 172.21.15.43:0/4183296002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f402806b710 msgr2=0x7f40280c9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:55.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.112+0000 7f402cbac700 1 --2- 172.21.15.43:0/4183296002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f402806b710 0x7f40280c9810 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4018009ba0 tx=0x7f4018004750 comp rx=0 tx=0).stop 2023-06-28T20:07:55.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.112+0000 7f402cbac700 1 -- 172.21.15.43:0/4183296002 shutdown_connections 2023-06-28T20:07:55.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.112+0000 7f402cbac700 1 --2- 172.21.15.43:0/4183296002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40280d0630 0x7f4028157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.112+0000 7f402cbac700 1 --2- 172.21.15.43:0/4183296002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f402806b710 0x7f40280c9810 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.112+0000 7f402cbac700 1 --2- 172.21.15.43:0/4183296002 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f400806ace0 0x7f400806d190 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.112+0000 7f402cbac700 1 --2- 172.21.15.43:0/4183296002 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f40280cbd40 0x7f4028157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.112+0000 7f402cbac700 1 -- 172.21.15.43:0/4183296002 >> 172.21.15.43:0/4183296002 conn(0x7f40280664b0 msgr2=0x7f40280c6800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:55.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.113+0000 7f402cbac700 1 -- 172.21.15.43:0/4183296002 shutdown_connections 2023-06-28T20:07:55.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.113+0000 7f402cbac700 1 -- 172.21.15.43:0/4183296002 wait complete. 2023-06-28T20:07:55.128 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-28T20:07:13.580969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-28T20:07:14.038358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-28T20:07:18.624759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-28T20:07:19.760017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-28T20:07:55.129 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-28T20:07:55.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.254+0000 7f0e4960d700 1 Processor -- start 2023-06-28T20:07:55.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.254+0000 7f0e4960d700 1 -- start start 2023-06-28T20:07:55.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.255+0000 7f0e4960d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e440d0a80 0x7f0e440d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:55.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.255+0000 7f0e4960d700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e440b89f0 con 0x7f0e440cf830 2023-06-28T20:07:55.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.255+0000 7f0e4960d700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e440d0f00 con 0x7f0e4415cf10 2023-06-28T20:07:55.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.255+0000 7f0e4960d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e440b8c80 con 0x7f0e440d0a80 2023-06-28T20:07:55.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.255+0000 7f0e42ffd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0e440cf830 0x7f0e440cfc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43248/0 (socket says 172.21.15.43:43248) 2023-06-28T20:07:55.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.255+0000 7f0e42ffd700 1 -- 172.21.15.43:0/2128291100 learned_addr learned my addr 172.21.15.43:0/2128291100 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:55.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.255+0000 7f0e427fc700 1 --2- 172.21.15.43:0/2128291100 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e440d0a80 0x7f0e440d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:55.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.255+0000 7f0e41ffb700 1 -- 172.21.15.43:0/2128291100 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2305141907 0 0) 0x7f0e440b89f0 con 0x7f0e440cf830 2023-06-28T20:07:55.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.255+0000 7f0e41ffb700 1 -- 172.21.15.43:0/2128291100 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e2c0035b0 con 0x7f0e440cf830 2023-06-28T20:07:55.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.255+0000 7f0e41ffb700 1 -- 172.21.15.43:0/2128291100 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3157669249 0 0) 0x7f0e2c0035b0 con 0x7f0e440cf830 2023-06-28T20:07:55.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.255+0000 7f0e41ffb700 1 -- 172.21.15.43:0/2128291100 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e440d0a80 msgr2=0x7f0e440d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:55.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.255+0000 7f0e41ffb700 1 --2- 172.21.15.43:0/2128291100 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e440d0a80 0x7f0e440d1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.255+0000 7f0e41ffb700 1 -- 172.21.15.43:0/2128291100 >> v1:172.21.15.178:6789/0 conn(0x7f0e4415cf10 legacy=0x7f0e440b82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:55.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.255+0000 7f0e41ffb700 1 -- 172.21.15.43:0/2128291100 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e440d11a0 con 0x7f0e440cf830 2023-06-28T20:07:55.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.256+0000 7f0e41ffb700 1 -- 172.21.15.43:0/2128291100 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f0e38001670 con 0x7f0e440cf830 2023-06-28T20:07:55.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.256+0000 7f0e41ffb700 1 -- 172.21.15.43:0/2128291100 >> v1:172.21.15.43:6789/0 conn(0x7f0e440cf830 legacy=0x7f0e440cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:55.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.256+0000 7f0e41ffb700 1 --2- 172.21.15.43:0/2128291100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e2c003a30 0x7f0e2c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:55.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.256+0000 7f0e41ffb700 1 --2- 172.21.15.43:0/2128291100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e2c0043d0 0x7f0e2c004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:55.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.256+0000 7f0e41ffb700 1 --2- 172.21.15.43:0/2128291100 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e440d0a80 0x7f0e2c0050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:55.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.256+0000 7f0e41ffb700 1 -- 172.21.15.43:0/2128291100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e440b8c80 con 0x7f0e2c003a30 2023-06-28T20:07:55.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.256+0000 7f0e41ffb700 1 -- 172.21.15.43:0/2128291100 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c00ba10 con 0x7f0e2c0043d0 2023-06-28T20:07:55.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.256+0000 7f0e41ffb700 1 -- 172.21.15.43:0/2128291100 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c00bba0 con 0x7f0e440d0a80 2023-06-28T20:07:55.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.256+0000 7f0e42ffd700 1 --2- 172.21.15.43:0/2128291100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e2c003a30 0x7f0e2c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:55.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.256+0000 7f0e427fc700 1 --2- 172.21.15.43:0/2128291100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e2c0043d0 0x7f0e2c004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:55.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.256+0000 7f0e437fe700 1 --2- 172.21.15.43:0/2128291100 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e440d0a80 0x7f0e2c0050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:55.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.256+0000 7f0e42ffd700 1 -- 172.21.15.43:0/2128291100 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e440d0a80 msgr2=0x7f0e2c0050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:55.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.256+0000 7f0e42ffd700 1 --2- 172.21.15.43:0/2128291100 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e440d0a80 0x7f0e2c0050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.256+0000 7f0e42ffd700 1 -- 172.21.15.43:0/2128291100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e2c0043d0 msgr2=0x7f0e2c004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:55.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.256+0000 7f0e42ffd700 1 --2- 172.21.15.43:0/2128291100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e2c0043d0 0x7f0e2c004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.256+0000 7f0e42ffd700 1 -- 172.21.15.43:0/2128291100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c0035b0 con 0x7f0e2c003a30 2023-06-28T20:07:55.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.256+0000 7f0e437fe700 1 --2- 172.21.15.43:0/2128291100 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e440d0a80 0x7f0e2c0050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:55.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.256+0000 7f0e42ffd700 1 --2- 172.21.15.43:0/2128291100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e2c003a30 0x7f0e2c003e00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800c2f0 tx=0x7f0e2c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c85475c7223702 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:55.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.256+0000 7f0e41ffb700 1 -- 172.21.15.43:0/2128291100 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3800ce30 con 0x7f0e2c003a30 2023-06-28T20:07:55.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.256+0000 7f0e41ffb700 1 -- 172.21.15.43:0/2128291100 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e38005960 con 0x7f0e2c003a30 2023-06-28T20:07:55.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.256+0000 7f0e41ffb700 1 -- 172.21.15.43:0/2128291100 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e38005cf0 con 0x7f0e2c003a30 2023-06-28T20:07:55.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.257+0000 7f0e4960d700 1 -- 172.21.15.43:0/2128291100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e2c003a30 msgr2=0x7f0e2c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:55.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.257+0000 7f0e4960d700 1 --2- 172.21.15.43:0/2128291100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e2c003a30 0x7f0e2c003e00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0e3800c2f0 tx=0x7f0e2c005880 comp rx=0 tx=0).stop 2023-06-28T20:07:55.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.257+0000 7f0e437fe700 1 -- 172.21.15.43:0/2128291100 reap_dead start 2023-06-28T20:07:55.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.257+0000 7f0e4960d700 1 -- 172.21.15.43:0/2128291100 shutdown_connections 2023-06-28T20:07:55.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.257+0000 7f0e4960d700 1 -- 172.21.15.43:0/2128291100 >> 172.21.15.43:0/2128291100 conn(0x7f0e4406a640 msgr2=0x7f0e4406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:55.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.257+0000 7f0e4960d700 1 -- 172.21.15.43:0/2128291100 shutdown_connections 2023-06-28T20:07:55.266 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.257+0000 7f0e4960d700 1 -- 172.21.15.43:0/2128291100 wait complete. 2023-06-28T20:07:55.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.258+0000 7f0e4960d700 1 Processor -- start 2023-06-28T20:07:55.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.258+0000 7f0e4960d700 1 -- start start 2023-06-28T20:07:55.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.259+0000 7f0e4960d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e2c003a30 0x7f0e440c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:55.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.259+0000 7f0e4960d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e44153730 0x7f0e44159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:55.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.259+0000 7f0e4960d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e44153ba0 0x7f0e4415a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:55.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.259+0000 7f0e4960d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e440c9810 con 0x7f0e2c003a30 2023-06-28T20:07:55.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.259+0000 7f0e4960d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e440cb5b0 con 0x7f0e44153730 2023-06-28T20:07:55.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.259+0000 7f0e4960d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4415e260 con 0x7f0e44153ba0 2023-06-28T20:07:55.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.259+0000 7f0e42ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e2c003a30 0x7f0e440c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:55.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.259+0000 7f0e42ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e2c003a30 0x7f0e440c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54074/0 (socket says 172.21.15.43:54074) 2023-06-28T20:07:55.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.259+0000 7f0e42ffd700 1 -- 172.21.15.43:0/2209037276 learned_addr learned my addr 172.21.15.43:0/2209037276 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:55.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.259+0000 7f0e437fe700 1 --2- 172.21.15.43:0/2209037276 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e44153ba0 0x7f0e4415a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:55.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.259+0000 7f0e427fc700 1 --2- 172.21.15.43:0/2209037276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e44153730 0x7f0e44159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:55.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.259+0000 7f0e42ffd700 1 -- 172.21.15.43:0/2209037276 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e44153ba0 msgr2=0x7f0e4415a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:55.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.259+0000 7f0e42ffd700 1 --2- 172.21.15.43:0/2209037276 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e44153ba0 0x7f0e4415a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.259+0000 7f0e42ffd700 1 -- 172.21.15.43:0/2209037276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e44153730 msgr2=0x7f0e44159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:55.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.259+0000 7f0e42ffd700 1 --2- 172.21.15.43:0/2209037276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e44153730 0x7f0e44159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.259+0000 7f0e42ffd700 1 -- 172.21.15.43:0/2209037276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4415e540 con 0x7f0e2c003a30 2023-06-28T20:07:55.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.260+0000 7f0e437fe700 1 --2- 172.21.15.43:0/2209037276 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e44153ba0 0x7f0e4415a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:55.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.260+0000 7f0e427fc700 1 --2- 172.21.15.43:0/2209037276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e44153730 0x7f0e44159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:55.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.260+0000 7f0e42ffd700 1 --2- 172.21.15.43:0/2209037276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e2c003a30 0x7f0e440c92d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0e38001930 tx=0x7f0e38001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:55.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.260+0000 7f0e2b7fe700 1 -- 172.21.15.43:0/2209037276 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e38001660 con 0x7f0e2c003a30 2023-06-28T20:07:55.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.260+0000 7f0e2b7fe700 1 -- 172.21.15.43:0/2209037276 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e38004a80 con 0x7f0e2c003a30 2023-06-28T20:07:55.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.260+0000 7f0e2b7fe700 1 -- 172.21.15.43:0/2209037276 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e38004e10 con 0x7f0e2c003a30 2023-06-28T20:07:55.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.260+0000 7f0e4960d700 1 -- 172.21.15.43:0/2209037276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4415e820 con 0x7f0e2c003a30 2023-06-28T20:07:55.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.260+0000 7f0e4960d700 1 -- 172.21.15.43:0/2209037276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4415de10 con 0x7f0e2c003a30 2023-06-28T20:07:55.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.261+0000 7f0e4960d700 1 -- 172.21.15.43:0/2209037276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e440cb6f0 con 0x7f0e2c003a30 2023-06-28T20:07:55.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.261+0000 7f0e2b7fe700 1 -- 172.21.15.43:0/2209037276 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0e380070d0 con 0x7f0e2c003a30 2023-06-28T20:07:55.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.262+0000 7f0e2b7fe700 1 --2- 172.21.15.43:0/2209037276 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0e3006ace0 0x7f0e3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:55.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.262+0000 7f0e2b7fe700 1 -- 172.21.15.43:0/2209037276 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0e3808ab80 con 0x7f0e2c003a30 2023-06-28T20:07:55.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.262+0000 7f0e427fc700 1 --2- 172.21.15.43:0/2209037276 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0e3006ace0 0x7f0e3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:55.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.262+0000 7f0e427fc700 1 --2- 172.21.15.43:0/2209037276 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0e3006ace0 0x7f0e3006d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0e4415ab00 tx=0x7f0e34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:55.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.266+0000 7f0e2b7fe700 1 -- 172.21.15.43:0/2209037276 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0e38055ea0 con 0x7f0e2c003a30 2023-06-28T20:07:55.467 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.464+0000 7f0e4960d700 1 -- 172.21.15.43:0/2209037276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0e440cb8c0 con 0x7f0e2c003a30 2023-06-28T20:07:55.468 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.465+0000 7f0e2b7fe700 1 -- 172.21.15.43:0/2209037276 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0e38017020 con 0x7f0e2c003a30 2023-06-28T20:07:55.468 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-06-28T20:07:55.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.467+0000 7f0e4960d700 1 -- 172.21.15.43:0/2209037276 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0e3006ace0 msgr2=0x7f0e3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:55.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.467+0000 7f0e4960d700 1 --2- 172.21.15.43:0/2209037276 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0e3006ace0 0x7f0e3006d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0e4415ab00 tx=0x7f0e34008040 comp rx=0 tx=0).stop 2023-06-28T20:07:55.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.467+0000 7f0e4960d700 1 -- 172.21.15.43:0/2209037276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e2c003a30 msgr2=0x7f0e440c92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:55.470 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.467+0000 7f0e4960d700 1 --2- 172.21.15.43:0/2209037276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e2c003a30 0x7f0e440c92d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0e38001930 tx=0x7f0e38001a00 comp rx=0 tx=0).stop 2023-06-28T20:07:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.467+0000 7f0e4960d700 1 -- 172.21.15.43:0/2209037276 shutdown_connections 2023-06-28T20:07:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.467+0000 7f0e4960d700 1 --2- 172.21.15.43:0/2209037276 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0e3006ace0 0x7f0e3006d190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.467+0000 7f0e4960d700 1 --2- 172.21.15.43:0/2209037276 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0e44153ba0 0x7f0e4415a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.467+0000 7f0e4960d700 1 --2- 172.21.15.43:0/2209037276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e2c003a30 0x7f0e440c92d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.467+0000 7f0e4960d700 1 --2- 172.21.15.43:0/2209037276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e44153730 0x7f0e44159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.467+0000 7f0e4960d700 1 -- 172.21.15.43:0/2209037276 >> 172.21.15.43:0/2209037276 conn(0x7f0e4406a640 msgr2=0x7f0e4415d330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:55.471 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.467+0000 7f0e4960d700 1 -- 172.21.15.43:0/2209037276 shutdown_connections 2023-06-28T20:07:55.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.467+0000 7f0e4960d700 1 -- 172.21.15.43:0/2209037276 wait complete. 2023-06-28T20:07:55.482 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-28T20:07:55.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.607+0000 7fd81f20f700 1 Processor -- start 2023-06-28T20:07:55.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.607+0000 7fd81f20f700 1 -- start start 2023-06-28T20:07:55.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.608+0000 7fd81f20f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd8180aec90 0x7fd818160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:55.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.608+0000 7fd81f20f700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd8180af060 0x7fd818070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:55.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.608+0000 7fd81f20f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd818160250 con 0x7fd81806ccc0 2023-06-28T20:07:55.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.608+0000 7fd81f20f700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd81815cb30 con 0x7fd8180aec90 2023-06-28T20:07:55.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.608+0000 7fd81f20f700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd818071410 con 0x7fd8180af060 2023-06-28T20:07:55.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.608+0000 7fd81d7ac700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd81806ccc0 0x7fd8180b1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43272/0 (socket says 172.21.15.43:43272) 2023-06-28T20:07:55.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.608+0000 7fd81d7ac700 1 -- 172.21.15.43:0/2312934342 learned_addr learned my addr 172.21.15.43:0/2312934342 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:55.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.608+0000 7fd81cfab700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd8180aec90 0x7fd818160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:55.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.608+0000 7fd817fff700 1 --2- 172.21.15.43:0/2312934342 >> v2:172.21.15.178:3300/0 conn(0x7fd8180af060 0x7fd818070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:55.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.608+0000 7fd81cfab700 1 -- 172.21.15.43:0/2312934342 >> v2:172.21.15.178:3300/0 conn(0x7fd8180af060 msgr2=0x7fd818070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:55.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.608+0000 7fd81cfab700 1 --2- 172.21.15.43:0/2312934342 >> v2:172.21.15.178:3300/0 conn(0x7fd8180af060 0x7fd818070bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.608+0000 7fd81cfab700 1 -- 172.21.15.43:0/2312934342 >> v1:172.21.15.43:6789/0 conn(0x7fd81806ccc0 legacy=0x7fd8180b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:55.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.608+0000 7fd81cfab700 1 -- 172.21.15.43:0/2312934342 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8181604f0 con 0x7fd8180aec90 2023-06-28T20:07:55.613 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.608+0000 7fd8177fe700 1 -- 172.21.15.43:0/2312934342 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604882184 0 0) 0x7fd818160250 con 0x7fd81806ccc0 2023-06-28T20:07:55.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.609+0000 7fd81cfab700 1 --2- 172.21.15.43:0/2312934342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8180aec90 0x7fd818160cf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd8180d0310 tx=0x7fd80c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5ef3965e19d467 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:55.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.609+0000 7fd8177fe700 1 -- 172.21.15.43:0/2312934342 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c011830 con 0x7fd8180aec90 2023-06-28T20:07:55.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.609+0000 7fd8177fe700 1 -- 172.21.15.43:0/2312934342 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80c011990 con 0x7fd8180aec90 2023-06-28T20:07:55.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.609+0000 7fd8177fe700 1 -- 172.21.15.43:0/2312934342 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c011d20 con 0x7fd8180aec90 2023-06-28T20:07:55.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.609+0000 7fd81f20f700 1 -- 172.21.15.43:0/2312934342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8180aec90 msgr2=0x7fd818160cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:55.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.609+0000 7fd81f20f700 1 --2- 172.21.15.43:0/2312934342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8180aec90 0x7fd818160cf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd8180d0310 tx=0x7fd80c009b20 comp rx=0 tx=0).stop 2023-06-28T20:07:55.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.609+0000 7fd81f20f700 1 -- 172.21.15.43:0/2312934342 shutdown_connections 2023-06-28T20:07:55.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.609+0000 7fd81f20f700 1 --2- 172.21.15.43:0/2312934342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8180aec90 0x7fd818160cf0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.609+0000 7fd81f20f700 1 --2- 172.21.15.43:0/2312934342 >> v2:172.21.15.178:3300/0 conn(0x7fd8180af060 0x7fd818070bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.609+0000 7fd81f20f700 1 -- 172.21.15.43:0/2312934342 >> 172.21.15.43:0/2312934342 conn(0x7fd81806b000 msgr2=0x7fd81806b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.609+0000 7fd81f20f700 1 -- 172.21.15.43:0/2312934342 shutdown_connections 2023-06-28T20:07:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.609+0000 7fd81f20f700 1 -- 172.21.15.43:0/2312934342 wait complete. 2023-06-28T20:07:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.610+0000 7fd81f20f700 1 Processor -- start 2023-06-28T20:07:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.611+0000 7fd81f20f700 1 -- start start 2023-06-28T20:07:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.611+0000 7fd81f20f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd81806ccc0 0x7fd818151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.611+0000 7fd81f20f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8180aec90 0x7fd8180caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.611+0000 7fd81f20f700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8180af060 0x7fd8180b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.611+0000 7fd81f20f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8180c4660 con 0x7fd81806ccc0 2023-06-28T20:07:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.611+0000 7fd81f20f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8180c4890 con 0x7fd8180aec90 2023-06-28T20:07:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.611+0000 7fd81f20f700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8180c4b70 con 0x7fd8180af060 2023-06-28T20:07:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.611+0000 7fd81cfab700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd81806ccc0 0x7fd818151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.612+0000 7fd81cfab700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd81806ccc0 0x7fd818151130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54088/0 (socket says 172.21.15.43:54088) 2023-06-28T20:07:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.612+0000 7fd81cfab700 1 -- 172.21.15.43:0/3094385428 learned_addr learned my addr 172.21.15.43:0/3094385428 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.612+0000 7fd817fff700 1 --2- 172.21.15.43:0/3094385428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8180aec90 0x7fd8180caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.612+0000 7fd81d7ac700 1 --2- 172.21.15.43:0/3094385428 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8180af060 0x7fd8180b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:55.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.612+0000 7fd81cfab700 1 -- 172.21.15.43:0/3094385428 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8180af060 msgr2=0x7fd8180b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:55.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.612+0000 7fd81cfab700 1 --2- 172.21.15.43:0/3094385428 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8180af060 0x7fd8180b05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.612+0000 7fd81cfab700 1 -- 172.21.15.43:0/3094385428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8180aec90 msgr2=0x7fd8180caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:55.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.612+0000 7fd81cfab700 1 --2- 172.21.15.43:0/3094385428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8180aec90 0x7fd8180caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.612+0000 7fd81cfab700 1 -- 172.21.15.43:0/3094385428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80c0094a0 con 0x7fd81806ccc0 2023-06-28T20:07:55.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.612+0000 7fd817fff700 1 --2- 172.21.15.43:0/3094385428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8180aec90 0x7fd8180caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:55.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.612+0000 7fd81cfab700 1 --2- 172.21.15.43:0/3094385428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd81806ccc0 0x7fd818151130 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd80c000c00 tx=0x7fd80c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.612+0000 7fd8157fa700 1 -- 172.21.15.43:0/3094385428 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c0036a0 con 0x7fd81806ccc0 2023-06-28T20:07:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.612+0000 7fd8157fa700 1 -- 172.21.15.43:0/3094385428 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd80c01f070 con 0x7fd81806ccc0 2023-06-28T20:07:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.612+0000 7fd8157fa700 1 -- 172.21.15.43:0/3094385428 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c003a60 con 0x7fd81806ccc0 2023-06-28T20:07:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.612+0000 7fd81f20f700 1 -- 172.21.15.43:0/3094385428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8180c4e50 con 0x7fd81806ccc0 2023-06-28T20:07:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.612+0000 7fd81f20f700 1 -- 172.21.15.43:0/3094385428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8180c5410 con 0x7fd81806ccc0 2023-06-28T20:07:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.613+0000 7fd81f20f700 1 -- 172.21.15.43:0/3094385428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8180728b0 con 0x7fd81806ccc0 2023-06-28T20:07:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.614+0000 7fd8157fa700 1 -- 172.21.15.43:0/3094385428 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd80c003bc0 con 0x7fd81806ccc0 2023-06-28T20:07:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.614+0000 7fd8157fa700 1 --2- 172.21.15.43:0/3094385428 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd7fc06ace0 0x7fd7fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:55.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.614+0000 7fd8157fa700 1 -- 172.21.15.43:0/3094385428 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd80c021070 con 0x7fd81806ccc0 2023-06-28T20:07:55.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.614+0000 7fd817fff700 1 --2- 172.21.15.43:0/3094385428 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd7fc06ace0 0x7fd7fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:55.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.615+0000 7fd817fff700 1 --2- 172.21.15.43:0/3094385428 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd7fc06ace0 0x7fd7fc06d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd8180adce0 tx=0x7fd804008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:55.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.618+0000 7fd8157fa700 1 -- 172.21.15.43:0/3094385428 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd80c055f40 con 0x7fd81806ccc0 2023-06-28T20:07:55.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.814+0000 7fd81f20f700 1 -- 172.21.15.43:0/3094385428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd81806d9c0 con 0x7fd81806ccc0 2023-06-28T20:07:55.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.815+0000 7fd8157fa700 1 -- 172.21.15.43:0/3094385428 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd80c059560 con 0x7fd81806ccc0 2023-06-28T20:07:55.818 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-06-28T20:07:55.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.817+0000 7fd81f20f700 1 -- 172.21.15.43:0/3094385428 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd7fc06ace0 msgr2=0x7fd7fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:55.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.817+0000 7fd81f20f700 1 --2- 172.21.15.43:0/3094385428 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd7fc06ace0 0x7fd7fc06d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd8180adce0 tx=0x7fd804008040 comp rx=0 tx=0).stop 2023-06-28T20:07:55.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.817+0000 7fd81f20f700 1 -- 172.21.15.43:0/3094385428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd81806ccc0 msgr2=0x7fd818151130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:55.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.817+0000 7fd81f20f700 1 --2- 172.21.15.43:0/3094385428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd81806ccc0 0x7fd818151130 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd80c000c00 tx=0x7fd80c004750 comp rx=0 tx=0).stop 2023-06-28T20:07:55.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.817+0000 7fd81f20f700 1 -- 172.21.15.43:0/3094385428 shutdown_connections 2023-06-28T20:07:55.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.817+0000 7fd81f20f700 1 --2- 172.21.15.43:0/3094385428 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd7fc06ace0 0x7fd7fc06d190 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.817+0000 7fd81f20f700 1 --2- 172.21.15.43:0/3094385428 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8180af060 0x7fd8180b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.817+0000 7fd81f20f700 1 --2- 172.21.15.43:0/3094385428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd81806ccc0 0x7fd818151130 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.817+0000 7fd81f20f700 1 --2- 172.21.15.43:0/3094385428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8180aec90 0x7fd8180caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.817+0000 7fd81f20f700 1 -- 172.21.15.43:0/3094385428 >> 172.21.15.43:0/3094385428 conn(0x7fd81806b000 msgr2=0x7fd81806b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:55.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.817+0000 7fd81f20f700 1 -- 172.21.15.43:0/3094385428 shutdown_connections 2023-06-28T20:07:55.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.817+0000 7fd81f20f700 1 -- 172.21.15.43:0/3094385428 wait complete. 2023-06-28T20:07:55.832 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-28T20:07:55.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.957+0000 7f147f0db700 1 Processor -- start 2023-06-28T20:07:55.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.957+0000 7f147f0db700 1 -- start start 2023-06-28T20:07:55.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.958+0000 7f147f0db700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f14780aec90 0x7f1478160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:55.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.958+0000 7f147f0db700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f147806ccc0 0x7f14780b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:55.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.958+0000 7f147f0db700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1478160250 con 0x7f14780af060 2023-06-28T20:07:55.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.958+0000 7f147f0db700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f147815cb30 con 0x7f147806ccc0 2023-06-28T20:07:55.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.958+0000 7f147f0db700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1478071410 con 0x7f14780aec90 2023-06-28T20:07:55.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.958+0000 7f147d678700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f147806ccc0 0x7f14780b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:55.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.958+0000 7f147d678700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f147806ccc0 0x7f14780b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54098/0 (socket says 172.21.15.43:54098) 2023-06-28T20:07:55.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.958+0000 7f147d678700 1 -- 172.21.15.43:0/752589240 learned_addr learned my addr 172.21.15.43:0/752589240 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:55.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.958+0000 7f147ce77700 1 --2- 172.21.15.43:0/752589240 >> v2:172.21.15.178:3300/0 conn(0x7f14780aec90 0x7f1478160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:55.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.958+0000 7f147d678700 1 -- 172.21.15.43:0/752589240 >> v2:172.21.15.178:3300/0 conn(0x7f14780aec90 msgr2=0x7f1478160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:55.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.958+0000 7f147d678700 1 --2- 172.21.15.43:0/752589240 >> v2:172.21.15.178:3300/0 conn(0x7f14780aec90 0x7f1478160cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.958+0000 7f147d678700 1 -- 172.21.15.43:0/752589240 >> v1:172.21.15.178:6789/0 conn(0x7f14780af060 legacy=0x7f1478070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:55.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.958+0000 7f147d678700 1 -- 172.21.15.43:0/752589240 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14781604f0 con 0x7f147806ccc0 2023-06-28T20:07:55.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.958+0000 7f147d678700 1 --2- 172.21.15.43:0/752589240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f147806ccc0 0x7f14780b1880 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f147806fce0 tx=0x7f147000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd4d1b6fcd70c30 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:55.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.959+0000 7f14777fe700 1 -- 172.21.15.43:0/752589240 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147000e070 con 0x7f147806ccc0 2023-06-28T20:07:55.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.959+0000 7f14777fe700 1 -- 172.21.15.43:0/752589240 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147000b9c0 con 0x7f147806ccc0 2023-06-28T20:07:55.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.959+0000 7f14777fe700 1 -- 172.21.15.43:0/752589240 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147000bd70 con 0x7f147806ccc0 2023-06-28T20:07:55.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.959+0000 7f147f0db700 1 -- 172.21.15.43:0/752589240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f147806ccc0 msgr2=0x7f14780b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:55.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.959+0000 7f147f0db700 1 --2- 172.21.15.43:0/752589240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f147806ccc0 0x7f14780b1880 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f147806fce0 tx=0x7f147000b330 comp rx=0 tx=0).stop 2023-06-28T20:07:55.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.959+0000 7f147f0db700 1 -- 172.21.15.43:0/752589240 shutdown_connections 2023-06-28T20:07:55.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.959+0000 7f147f0db700 1 --2- 172.21.15.43:0/752589240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f147806ccc0 0x7f14780b1880 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.959+0000 7f147f0db700 1 --2- 172.21.15.43:0/752589240 >> v2:172.21.15.178:3300/0 conn(0x7f14780aec90 0x7f1478160cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.959+0000 7f147f0db700 1 -- 172.21.15.43:0/752589240 >> 172.21.15.43:0/752589240 conn(0x7f147806b000 msgr2=0x7f147806b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:55.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.959+0000 7f147f0db700 1 -- 172.21.15.43:0/752589240 shutdown_connections 2023-06-28T20:07:55.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.959+0000 7f147f0db700 1 -- 172.21.15.43:0/752589240 wait complete. 2023-06-28T20:07:55.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.961+0000 7f147f0db700 1 Processor -- start 2023-06-28T20:07:55.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.961+0000 7f147f0db700 1 -- start start 2023-06-28T20:07:55.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.962+0000 7f147f0db700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f147806ccc0 0x7f1478159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:55.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.962+0000 7f147f0db700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14780aec90 0x7f147815e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:55.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.962+0000 7f147f0db700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14780af060 0x7f14780cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:55.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.962+0000 7f147f0db700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147815f4b0 con 0x7f14780af060 2023-06-28T20:07:55.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.962+0000 7f147f0db700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147815eef0 con 0x7f14780aec90 2023-06-28T20:07:55.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.962+0000 7f147f0db700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f147815f1d0 con 0x7f147806ccc0 2023-06-28T20:07:55.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.962+0000 7f147d678700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14780af060 0x7f14780cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:55.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.962+0000 7f147d678700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14780af060 0x7f14780cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54104/0 (socket says 172.21.15.43:54104) 2023-06-28T20:07:55.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.962+0000 7f147d678700 1 -- 172.21.15.43:0/4088592448 learned_addr learned my addr 172.21.15.43:0/4088592448 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:55.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.962+0000 7f147ce77700 1 --2- 172.21.15.43:0/4088592448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f147806ccc0 0x7f1478159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:55.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.962+0000 7f1477fff700 1 --2- 172.21.15.43:0/4088592448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14780aec90 0x7f147815e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:55.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.962+0000 7f147d678700 1 -- 172.21.15.43:0/4088592448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f147806ccc0 msgr2=0x7f1478159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:55.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.962+0000 7f147d678700 1 --2- 172.21.15.43:0/4088592448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f147806ccc0 0x7f1478159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.962+0000 7f147d678700 1 -- 172.21.15.43:0/4088592448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14780aec90 msgr2=0x7f147815e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:55.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.962+0000 7f147d678700 1 --2- 172.21.15.43:0/4088592448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14780aec90 0x7f147815e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:55.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.962+0000 7f147d678700 1 -- 172.21.15.43:0/4088592448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1470009b30 con 0x7f14780af060 2023-06-28T20:07:55.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.962+0000 7f147ce77700 1 --2- 172.21.15.43:0/4088592448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f147806ccc0 0x7f1478159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:55.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.962+0000 7f1477fff700 1 --2- 172.21.15.43:0/4088592448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14780aec90 0x7f147815e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:55.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.963+0000 7f147d678700 1 --2- 172.21.15.43:0/4088592448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14780af060 0x7f14780cb5a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f14780b2610 tx=0x7f1470004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:55.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.963+0000 7f14757fa700 1 -- 172.21.15.43:0/4088592448 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14700092e0 con 0x7f14780af060 2023-06-28T20:07:55.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.963+0000 7f14757fa700 1 -- 172.21.15.43:0/4088592448 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147001c440 con 0x7f14780af060 2023-06-28T20:07:55.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.963+0000 7f147f0db700 1 -- 172.21.15.43:0/4088592448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14781f3610 con 0x7f14780af060 2023-06-28T20:07:55.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.963+0000 7f14757fa700 1 -- 172.21.15.43:0/4088592448 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147001c7f0 con 0x7f14780af060 2023-06-28T20:07:55.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.963+0000 7f147f0db700 1 -- 172.21.15.43:0/4088592448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f147815eab0 con 0x7f14780af060 2023-06-28T20:07:55.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.964+0000 7f147f0db700 1 -- 172.21.15.43:0/4088592448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14781f3470 con 0x7f14780af060 2023-06-28T20:07:55.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.964+0000 7f14757fa700 1 -- 172.21.15.43:0/4088592448 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f147001c950 con 0x7f14780af060 2023-06-28T20:07:55.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.965+0000 7f14757fa700 1 --2- 172.21.15.43:0/4088592448 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f145c06ace0 0x7f145c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:55.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.965+0000 7f14757fa700 1 -- 172.21.15.43:0/4088592448 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f147008b0f0 con 0x7f14780af060 2023-06-28T20:07:55.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.965+0000 7f147ce77700 1 --2- 172.21.15.43:0/4088592448 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f145c06ace0 0x7f145c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:55.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.965+0000 7f147ce77700 1 --2- 172.21.15.43:0/4088592448 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f145c06ace0 0x7f145c06d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f14781e7d10 tx=0x7f146c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:55.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:55.969+0000 7f14757fa700 1 -- 172.21.15.43:0/4088592448 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f14781f3470 con 0x7f14780af060 2023-06-28T20:07:56.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.169+0000 7f147f0db700 1 -- 172.21.15.43:0/4088592448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f14781f3470 con 0x7f14780af060 2023-06-28T20:07:56.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.170+0000 7f14757fa700 1 -- 172.21.15.43:0/4088592448 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f14781f3470 con 0x7f14780af060 2023-06-28T20:07:56.173 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-06-28T20:07:56.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.172+0000 7f147f0db700 1 -- 172.21.15.43:0/4088592448 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f145c06ace0 msgr2=0x7f145c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:56.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.172+0000 7f147f0db700 1 --2- 172.21.15.43:0/4088592448 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f145c06ace0 0x7f145c06d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f14781e7d10 tx=0x7f146c008040 comp rx=0 tx=0).stop 2023-06-28T20:07:56.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.172+0000 7f147f0db700 1 -- 172.21.15.43:0/4088592448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14780af060 msgr2=0x7f14780cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:56.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.172+0000 7f147f0db700 1 --2- 172.21.15.43:0/4088592448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14780af060 0x7f14780cb5a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f14780b2610 tx=0x7f1470004ad0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.172+0000 7f147f0db700 1 -- 172.21.15.43:0/4088592448 shutdown_connections 2023-06-28T20:07:56.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.172+0000 7f147f0db700 1 --2- 172.21.15.43:0/4088592448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14780af060 0x7f14780cb5a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.172+0000 7f147f0db700 1 --2- 172.21.15.43:0/4088592448 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f145c06ace0 0x7f145c06d190 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.172+0000 7f147f0db700 1 --2- 172.21.15.43:0/4088592448 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f147806ccc0 0x7f1478159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.172+0000 7f147f0db700 1 --2- 172.21.15.43:0/4088592448 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f14780aec90 0x7f147815e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.172+0000 7f147f0db700 1 -- 172.21.15.43:0/4088592448 >> 172.21.15.43:0/4088592448 conn(0x7f147806b000 msgr2=0x7f14780ae170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:56.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.172+0000 7f147f0db700 1 -- 172.21.15.43:0/4088592448 shutdown_connections 2023-06-28T20:07:56.177 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.172+0000 7f147f0db700 1 -- 172.21.15.43:0/4088592448 wait complete. 2023-06-28T20:07:56.186 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-28T20:07:56.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.311+0000 7f7b7b0ee700 1 Processor -- start 2023-06-28T20:07:56.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.311+0000 7f7b7b0ee700 1 -- start start 2023-06-28T20:07:56.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.312+0000 7f7b7b0ee700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b7413c8b0 0x7f7b740cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:56.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.312+0000 7f7b7b0ee700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b7415a770 con 0x7f7b7413ef30 2023-06-28T20:07:56.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.312+0000 7f7b7b0ee700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b7413cd00 con 0x7f7b740ce8a0 2023-06-28T20:07:56.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.312+0000 7f7b7b0ee700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b740cf1b0 con 0x7f7b7413c8b0 2023-06-28T20:07:56.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.312+0000 7f7b7968b700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7b7413ef30 0x7f7b7413bb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43298/0 (socket says 172.21.15.43:43298) 2023-06-28T20:07:56.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.312+0000 7f7b7968b700 1 -- 172.21.15.43:0/725778291 learned_addr learned my addr 172.21.15.43:0/725778291 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:56.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.312+0000 7f7b73fff700 1 --2- 172.21.15.43:0/725778291 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b7413c8b0 0x7f7b740cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:56.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.312+0000 7f7b737fe700 1 -- 172.21.15.43:0/725778291 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2384096298 0 0) 0x7f7b7415a770 con 0x7f7b7413ef30 2023-06-28T20:07:56.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.312+0000 7f7b737fe700 1 -- 172.21.15.43:0/725778291 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b640035b0 con 0x7f7b7413ef30 2023-06-28T20:07:56.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.312+0000 7f7b737fe700 1 -- 172.21.15.43:0/725778291 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 209141746 0 0) 0x7f7b640035b0 con 0x7f7b7413ef30 2023-06-28T20:07:56.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.312+0000 7f7b737fe700 1 -- 172.21.15.43:0/725778291 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b7413c8b0 msgr2=0x7f7b740cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:56.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.312+0000 7f7b737fe700 1 --2- 172.21.15.43:0/725778291 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b7413c8b0 0x7f7b740cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.312+0000 7f7b737fe700 1 -- 172.21.15.43:0/725778291 >> v1:172.21.15.178:6789/0 conn(0x7f7b740ce8a0 legacy=0x7f7b7415d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:56.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.312+0000 7f7b737fe700 1 -- 172.21.15.43:0/725778291 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b74159b60 con 0x7f7b7413ef30 2023-06-28T20:07:56.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.312+0000 7f7b737fe700 1 -- 172.21.15.43:0/725778291 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f7b6c0019f0 con 0x7f7b7413ef30 2023-06-28T20:07:56.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.312+0000 7f7b737fe700 1 -- 172.21.15.43:0/725778291 >> v1:172.21.15.43:6789/0 conn(0x7f7b7413ef30 legacy=0x7f7b7413bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:56.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.313+0000 7f7b737fe700 1 --2- 172.21.15.43:0/725778291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b64003a30 0x7f7b64003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:56.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.313+0000 7f7b737fe700 1 --2- 172.21.15.43:0/725778291 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b7413c8b0 0x7f7b64004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:56.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.313+0000 7f7b737fe700 1 --2- 172.21.15.43:0/725778291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b64004d90 0x7f7b64005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:56.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.313+0000 7f7b737fe700 1 -- 172.21.15.43:0/725778291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b740cf1b0 con 0x7f7b64003a30 2023-06-28T20:07:56.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.313+0000 7f7b737fe700 1 -- 172.21.15.43:0/725778291 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6400ba10 con 0x7f7b64004d90 2023-06-28T20:07:56.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.313+0000 7f7b737fe700 1 -- 172.21.15.43:0/725778291 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6400bba0 con 0x7f7b7413c8b0 2023-06-28T20:07:56.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.313+0000 7f7b78e8a700 1 --2- 172.21.15.43:0/725778291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b64003a30 0x7f7b64003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:56.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.313+0000 7f7b7968b700 1 --2- 172.21.15.43:0/725778291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b64004d90 0x7f7b64005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:56.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.313+0000 7f7b73fff700 1 --2- 172.21.15.43:0/725778291 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b7413c8b0 0x7f7b64004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:56.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.313+0000 7f7b78e8a700 1 -- 172.21.15.43:0/725778291 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b7413c8b0 msgr2=0x7f7b64004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:56.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.313+0000 7f7b78e8a700 1 --2- 172.21.15.43:0/725778291 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b7413c8b0 0x7f7b64004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.313+0000 7f7b78e8a700 1 -- 172.21.15.43:0/725778291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b64004d90 msgr2=0x7f7b64005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:56.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.313+0000 7f7b78e8a700 1 --2- 172.21.15.43:0/725778291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b64004d90 0x7f7b64005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.313+0000 7f7b78e8a700 1 -- 172.21.15.43:0/725778291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b640035b0 con 0x7f7b64003a30 2023-06-28T20:07:56.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.313+0000 7f7b78e8a700 1 --2- 172.21.15.43:0/725778291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b64003a30 0x7f7b64003e00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7b68001930 tx=0x7f7b64005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12aa591e9100310 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:56.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.313+0000 7f7b737fe700 1 -- 172.21.15.43:0/725778291 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b680013a0 con 0x7f7b64003a30 2023-06-28T20:07:56.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.313+0000 7f7b737fe700 1 -- 172.21.15.43:0/725778291 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6800c690 con 0x7f7b64003a30 2023-06-28T20:07:56.322 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.313+0000 7f7b737fe700 1 -- 172.21.15.43:0/725778291 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6800ca40 con 0x7f7b64003a30 2023-06-28T20:07:56.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.314+0000 7f7b7b0ee700 1 -- 172.21.15.43:0/725778291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b64003a30 msgr2=0x7f7b64003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:56.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.314+0000 7f7b7b0ee700 1 --2- 172.21.15.43:0/725778291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b64003a30 0x7f7b64003e00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7b68001930 tx=0x7f7b64005880 comp rx=0 tx=0).stop 2023-06-28T20:07:56.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.314+0000 7f7b7968b700 1 -- 172.21.15.43:0/725778291 reap_dead start 2023-06-28T20:07:56.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.314+0000 7f7b7b0ee700 1 -- 172.21.15.43:0/725778291 shutdown_connections 2023-06-28T20:07:56.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.314+0000 7f7b7b0ee700 1 -- 172.21.15.43:0/725778291 >> 172.21.15.43:0/725778291 conn(0x7f7b74069ef0 msgr2=0x7f7b7406a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:56.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.314+0000 7f7b7b0ee700 1 -- 172.21.15.43:0/725778291 shutdown_connections 2023-06-28T20:07:56.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.314+0000 7f7b7b0ee700 1 -- 172.21.15.43:0/725778291 wait complete. 2023-06-28T20:07:56.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.316+0000 7f7b7b0ee700 1 Processor -- start 2023-06-28T20:07:56.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.316+0000 7f7b7b0ee700 1 -- start start 2023-06-28T20:07:56.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.317+0000 7f7b7b0ee700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b7413a4b0 0x7f7b74142760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:56.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.317+0000 7f7b7b0ee700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b7413a880 0x7f7b7413f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:56.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.317+0000 7f7b7b0ee700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b74153820 0x7f7b7413d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:56.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.317+0000 7f7b7b0ee700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b741422d0 con 0x7f7b74153820 2023-06-28T20:07:56.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.317+0000 7f7b7b0ee700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b74141700 con 0x7f7b7413a4b0 2023-06-28T20:07:56.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.317+0000 7f7b7b0ee700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b741419a0 con 0x7f7b7413a880 2023-06-28T20:07:56.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.317+0000 7f7b7968b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b74153820 0x7f7b7413d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:56.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.317+0000 7f7b7968b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b74153820 0x7f7b7413d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54124/0 (socket says 172.21.15.43:54124) 2023-06-28T20:07:56.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.317+0000 7f7b7968b700 1 -- 172.21.15.43:0/1315908584 learned_addr learned my addr 172.21.15.43:0/1315908584 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.317+0000 7f7b73fff700 1 --2- 172.21.15.43:0/1315908584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b7413a880 0x7f7b7413f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.317+0000 7f7b78e8a700 1 --2- 172.21.15.43:0/1315908584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b7413a4b0 0x7f7b74142760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.317+0000 7f7b7968b700 1 -- 172.21.15.43:0/1315908584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b7413a880 msgr2=0x7f7b7413f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.317+0000 7f7b7968b700 1 --2- 172.21.15.43:0/1315908584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b7413a880 0x7f7b7413f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.317+0000 7f7b7968b700 1 -- 172.21.15.43:0/1315908584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b7413a4b0 msgr2=0x7f7b74142760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.317+0000 7f7b7968b700 1 --2- 172.21.15.43:0/1315908584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b7413a4b0 0x7f7b74142760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.317+0000 7f7b7968b700 1 -- 172.21.15.43:0/1315908584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b74141c80 con 0x7f7b74153820 2023-06-28T20:07:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.317+0000 7f7b78e8a700 1 --2- 172.21.15.43:0/1315908584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b7413a4b0 0x7f7b74142760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.317+0000 7f7b7968b700 1 --2- 172.21.15.43:0/1315908584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b74153820 0x7f7b7413d4b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c001ee0 tx=0x7f7b6c00a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.317+0000 7f7b717fa700 1 -- 172.21.15.43:0/1315908584 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c00ad10 con 0x7f7b74153820 2023-06-28T20:07:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.317+0000 7f7b717fa700 1 -- 172.21.15.43:0/1315908584 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6c0133f0 con 0x7f7b74153820 2023-06-28T20:07:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.317+0000 7f7b7b0ee700 1 -- 172.21.15.43:0/1315908584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b74141f60 con 0x7f7b74153820 2023-06-28T20:07:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.317+0000 7f7b717fa700 1 -- 172.21.15.43:0/1315908584 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c0137a0 con 0x7f7b74153820 2023-06-28T20:07:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.318+0000 7f7b7b0ee700 1 -- 172.21.15.43:0/1315908584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b7413d9f0 con 0x7f7b74153820 2023-06-28T20:07:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.318+0000 7f7b7b0ee700 1 -- 172.21.15.43:0/1315908584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b740728b0 con 0x7f7b74153820 2023-06-28T20:07:56.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.319+0000 7f7b717fa700 1 -- 172.21.15.43:0/1315908584 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7b6c013900 con 0x7f7b74153820 2023-06-28T20:07:56.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.319+0000 7f7b717fa700 1 --2- 172.21.15.43:0/1315908584 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7b5806ace0 0x7f7b5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:56.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.319+0000 7f7b717fa700 1 -- 172.21.15.43:0/1315908584 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7b6c089cc0 con 0x7f7b74153820 2023-06-28T20:07:56.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.319+0000 7f7b78e8a700 1 --2- 172.21.15.43:0/1315908584 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7b5806ace0 0x7f7b5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:56.331 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.320+0000 7f7b78e8a700 1 --2- 172.21.15.43:0/1315908584 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7b5806ace0 0x7f7b5806d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7b7406cc00 tx=0x7f7b6800a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:56.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.324+0000 7f7b717fa700 1 -- 172.21.15.43:0/1315908584 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7b6c054e20 con 0x7f7b74153820 2023-06-28T20:07:56.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.520+0000 7f7b7b0ee700 1 -- 172.21.15.43:0/1315908584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7b7406a3c0 con 0x7f7b74153820 2023-06-28T20:07:56.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.520+0000 7f7b717fa700 1 -- 172.21.15.43:0/1315908584 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7b6c058440 con 0x7f7b74153820 2023-06-28T20:07:56.524 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-06-28T20:07:56.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.523+0000 7f7b7b0ee700 1 -- 172.21.15.43:0/1315908584 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7b5806ace0 msgr2=0x7f7b5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:56.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.523+0000 7f7b7b0ee700 1 --2- 172.21.15.43:0/1315908584 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7b5806ace0 0x7f7b5806d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7b7406cc00 tx=0x7f7b6800a000 comp rx=0 tx=0).stop 2023-06-28T20:07:56.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.523+0000 7f7b7b0ee700 1 -- 172.21.15.43:0/1315908584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b74153820 msgr2=0x7f7b7413d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:56.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.523+0000 7f7b7b0ee700 1 --2- 172.21.15.43:0/1315908584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b74153820 0x7f7b7413d4b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c001ee0 tx=0x7f7b6c00a600 comp rx=0 tx=0).stop 2023-06-28T20:07:56.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.523+0000 7f7b7b0ee700 1 -- 172.21.15.43:0/1315908584 shutdown_connections 2023-06-28T20:07:56.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.523+0000 7f7b7b0ee700 1 --2- 172.21.15.43:0/1315908584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b74153820 0x7f7b7413d4b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.523+0000 7f7b7b0ee700 1 --2- 172.21.15.43:0/1315908584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b7413a4b0 0x7f7b74142760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.523+0000 7f7b7b0ee700 1 --2- 172.21.15.43:0/1315908584 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7b5806ace0 0x7f7b5806d190 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.523+0000 7f7b7b0ee700 1 --2- 172.21.15.43:0/1315908584 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7b7413a880 0x7f7b7413f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.523+0000 7f7b7b0ee700 1 -- 172.21.15.43:0/1315908584 >> 172.21.15.43:0/1315908584 conn(0x7f7b74069ef0 msgr2=0x7f7b7413bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:56.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.523+0000 7f7b7b0ee700 1 -- 172.21.15.43:0/1315908584 shutdown_connections 2023-06-28T20:07:56.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.523+0000 7f7b7b0ee700 1 -- 172.21.15.43:0/1315908584 wait complete. 2023-06-28T20:07:56.538 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-28T20:07:56.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.660+0000 7fef2c114700 1 Processor -- start 2023-06-28T20:07:56.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.660+0000 7fef2c114700 1 -- start start 2023-06-28T20:07:56.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.661+0000 7fef2c114700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef241592d0 0x7fef240b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:56.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.661+0000 7fef2c114700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fef241596a0 0x7fef240b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:56.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.661+0000 7fef2c114700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef24070b40 con 0x7fef24071900 2023-06-28T20:07:56.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.661+0000 7fef2c114700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef24070e20 con 0x7fef241596a0 2023-06-28T20:07:56.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.661+0000 7fef2c114700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef24071100 con 0x7fef241592d0 2023-06-28T20:07:56.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.661+0000 7fef2a6b1700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fef24071900 0x7fef2415d070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43318/0 (socket says 172.21.15.43:43318) 2023-06-28T20:07:56.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.661+0000 7fef2a6b1700 1 -- 172.21.15.43:0/4201983964 learned_addr learned my addr 172.21.15.43:0/4201983964 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:56.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.661+0000 7fef28eae700 1 -- 172.21.15.43:0/4201983964 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2848127546 0 0) 0x7fef24070b40 con 0x7fef24071900 2023-06-28T20:07:56.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.661+0000 7fef29eb0700 1 --2- 172.21.15.43:0/4201983964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef241592d0 0x7fef240b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:56.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.661+0000 7fef28eae700 1 -- 172.21.15.43:0/4201983964 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef100035b0 con 0x7fef24071900 2023-06-28T20:07:56.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.661+0000 7fef296af700 1 --2- 172.21.15.43:0/4201983964 >> v2:172.21.15.178:3300/0 conn(0x7fef241596a0 0x7fef240b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:56.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.662+0000 7fef28eae700 1 -- 172.21.15.43:0/4201983964 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 30272843 0 0) 0x7fef100035b0 con 0x7fef24071900 2023-06-28T20:07:56.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.662+0000 7fef28eae700 1 -- 172.21.15.43:0/4201983964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef241592d0 msgr2=0x7fef240b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:56.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.662+0000 7fef28eae700 1 --2- 172.21.15.43:0/4201983964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef241592d0 0x7fef240b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.662+0000 7fef28eae700 1 -- 172.21.15.43:0/4201983964 >> v2:172.21.15.178:3300/0 conn(0x7fef241596a0 msgr2=0x7fef240b1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.662+0000 7fef28eae700 1 --2- 172.21.15.43:0/4201983964 >> v2:172.21.15.178:3300/0 conn(0x7fef241596a0 0x7fef240b1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.662+0000 7fef28eae700 1 -- 172.21.15.43:0/4201983964 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef240713e0 con 0x7fef24071900 2023-06-28T20:07:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.662+0000 7fef28eae700 1 -- 172.21.15.43:0/4201983964 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7fef1c0019f0 con 0x7fef24071900 2023-06-28T20:07:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.662+0000 7fef28eae700 1 -- 172.21.15.43:0/4201983964 >> v1:172.21.15.43:6789/0 conn(0x7fef24071900 legacy=0x7fef2415d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.662+0000 7fef28eae700 1 --2- 172.21.15.43:0/4201983964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef10003a00 0x7fef10003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.662+0000 7fef28eae700 1 --2- 172.21.15.43:0/4201983964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef10004430 0x7fef100048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.662+0000 7fef28eae700 1 --2- 172.21.15.43:0/4201983964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef241592d0 0x7fef10004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:56.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.662+0000 7fef28eae700 1 -- 172.21.15.43:0/4201983964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef24070e20 con 0x7fef10004430 2023-06-28T20:07:56.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.662+0000 7fef28eae700 1 -- 172.21.15.43:0/4201983964 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef24071100 con 0x7fef10003a00 2023-06-28T20:07:56.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.662+0000 7fef28eae700 1 -- 172.21.15.43:0/4201983964 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1000ba10 con 0x7fef241592d0 2023-06-28T20:07:56.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.662+0000 7fef296af700 1 --2- 172.21.15.43:0/4201983964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef10004430 0x7fef100048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:56.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.662+0000 7fef29eb0700 1 --2- 172.21.15.43:0/4201983964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef10003a00 0x7fef10003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:56.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.662+0000 7fef2a6b1700 1 --2- 172.21.15.43:0/4201983964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef241592d0 0x7fef10004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:56.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.662+0000 7fef296af700 1 -- 172.21.15.43:0/4201983964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef241592d0 msgr2=0x7fef10004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:56.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.662+0000 7fef296af700 1 --2- 172.21.15.43:0/4201983964 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef241592d0 0x7fef10004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.662+0000 7fef296af700 1 -- 172.21.15.43:0/4201983964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef10003a00 msgr2=0x7fef10003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:56.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.662+0000 7fef296af700 1 --2- 172.21.15.43:0/4201983964 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef10003a00 0x7fef10003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.662+0000 7fef296af700 1 -- 172.21.15.43:0/4201983964 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef100035b0 con 0x7fef10004430 2023-06-28T20:07:56.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.662+0000 7fef296af700 1 --2- 172.21.15.43:0/4201983964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef10004430 0x7fef100048f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fef20009b30 tx=0x7fef10005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db54ff64e545f3aa server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:56.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.663+0000 7fef28eae700 1 -- 172.21.15.43:0/4201983964 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2000e6c0 con 0x7fef10004430 2023-06-28T20:07:56.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.663+0000 7fef28eae700 1 -- 172.21.15.43:0/4201983964 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2000e820 con 0x7fef10004430 2023-06-28T20:07:56.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.663+0000 7fef28eae700 1 -- 172.21.15.43:0/4201983964 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2000eb90 con 0x7fef10004430 2023-06-28T20:07:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.663+0000 7fef2c114700 1 -- 172.21.15.43:0/4201983964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef10004430 msgr2=0x7fef100048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.663+0000 7fef2c114700 1 --2- 172.21.15.43:0/4201983964 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef10004430 0x7fef100048f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fef20009b30 tx=0x7fef10005880 comp rx=0 tx=0).stop 2023-06-28T20:07:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.663+0000 7fef2a6b1700 1 -- 172.21.15.43:0/4201983964 reap_dead start 2023-06-28T20:07:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.663+0000 7fef2c114700 1 -- 172.21.15.43:0/4201983964 shutdown_connections 2023-06-28T20:07:56.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.663+0000 7fef2c114700 1 -- 172.21.15.43:0/4201983964 >> 172.21.15.43:0/4201983964 conn(0x7fef2406b000 msgr2=0x7fef2406b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.663+0000 7fef2c114700 1 -- 172.21.15.43:0/4201983964 shutdown_connections 2023-06-28T20:07:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.663+0000 7fef2c114700 1 -- 172.21.15.43:0/4201983964 wait complete. 2023-06-28T20:07:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.664+0000 7fef2c114700 1 Processor -- start 2023-06-28T20:07:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.664+0000 7fef2c114700 1 -- start start 2023-06-28T20:07:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.665+0000 7fef2c114700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef10004430 0x7fef240b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.665+0000 7fef2c114700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef2414f3c0 0x7fef2415b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.665+0000 7fef2c114700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef240ce110 0x7fef240c3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.665+0000 7fef2c114700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2415c190 con 0x7fef240ce110 2023-06-28T20:07:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.665+0000 7fef2c114700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2414f100 con 0x7fef10004430 2023-06-28T20:07:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.665+0000 7fef2c114700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2415c930 con 0x7fef2414f3c0 2023-06-28T20:07:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.665+0000 7fef2a6b1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef240ce110 0x7fef240c3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.665+0000 7fef2a6b1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef240ce110 0x7fef240c3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54144/0 (socket says 172.21.15.43:54144) 2023-06-28T20:07:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.665+0000 7fef2a6b1700 1 -- 172.21.15.43:0/1952747771 learned_addr learned my addr 172.21.15.43:0/1952747771 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.665+0000 7fef296af700 1 --2- 172.21.15.43:0/1952747771 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef2414f3c0 0x7fef2415b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.665+0000 7fef29eb0700 1 --2- 172.21.15.43:0/1952747771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef10004430 0x7fef240b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:56.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.666+0000 7fef2a6b1700 1 -- 172.21.15.43:0/1952747771 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef2414f3c0 msgr2=0x7fef2415b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.666+0000 7fef2a6b1700 1 --2- 172.21.15.43:0/1952747771 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef2414f3c0 0x7fef2415b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.666+0000 7fef2a6b1700 1 -- 172.21.15.43:0/1952747771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef10004430 msgr2=0x7fef240b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.666+0000 7fef2a6b1700 1 --2- 172.21.15.43:0/1952747771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef10004430 0x7fef240b2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.666+0000 7fef2a6b1700 1 -- 172.21.15.43:0/1952747771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2415c370 con 0x7fef240ce110 2023-06-28T20:07:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.666+0000 7fef296af700 1 --2- 172.21.15.43:0/1952747771 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef2414f3c0 0x7fef2415b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.666+0000 7fef29eb0700 1 --2- 172.21.15.43:0/1952747771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef10004430 0x7fef240b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.666+0000 7fef2a6b1700 1 --2- 172.21.15.43:0/1952747771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef240ce110 0x7fef240c3620 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fef1c001330 tx=0x7fef1c00a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.666+0000 7fef1a7fc700 1 -- 172.21.15.43:0/1952747771 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c0133f0 con 0x7fef240ce110 2023-06-28T20:07:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.666+0000 7fef1a7fc700 1 -- 172.21.15.43:0/1952747771 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1c013590 con 0x7fef240ce110 2023-06-28T20:07:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.666+0000 7fef2c114700 1 -- 172.21.15.43:0/1952747771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef2415c650 con 0x7fef240ce110 2023-06-28T20:07:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.666+0000 7fef1a7fc700 1 -- 172.21.15.43:0/1952747771 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1c013940 con 0x7fef240ce110 2023-06-28T20:07:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.666+0000 7fef2c114700 1 -- 172.21.15.43:0/1952747771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef2414e920 con 0x7fef240ce110 2023-06-28T20:07:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.667+0000 7fef2c114700 1 -- 172.21.15.43:0/1952747771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef240cc3e0 con 0x7fef240ce110 2023-06-28T20:07:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.667+0000 7fef1a7fc700 1 -- 172.21.15.43:0/1952747771 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fef1c013aa0 con 0x7fef240ce110 2023-06-28T20:07:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.668+0000 7fef1a7fc700 1 --2- 172.21.15.43:0/1952747771 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fef0806ace0 0x7fef0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:56.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.668+0000 7fef29eb0700 1 --2- 172.21.15.43:0/1952747771 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fef0806ace0 0x7fef0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:56.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.668+0000 7fef1a7fc700 1 -- 172.21.15.43:0/1952747771 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fef1c089da0 con 0x7fef240ce110 2023-06-28T20:07:56.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.668+0000 7fef29eb0700 1 --2- 172.21.15.43:0/1952747771 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fef0806ace0 0x7fef0806d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fef14009b80 tx=0x7fef14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:56.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.672+0000 7fef1a7fc700 1 -- 172.21.15.43:0/1952747771 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fef1c054e50 con 0x7fef240ce110 2023-06-28T20:07:56.894 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.891+0000 7fef2c114700 1 -- 172.21.15.43:0/1952747771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fef2414eab0 con 0x7fef240ce110 2023-06-28T20:07:56.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.892+0000 7fef1a7fc700 1 -- 172.21.15.43:0/1952747771 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fef1c058470 con 0x7fef240ce110 2023-06-28T20:07:56.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.894+0000 7fef2c114700 1 -- 172.21.15.43:0/1952747771 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fef0806ace0 msgr2=0x7fef0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:56.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.894+0000 7fef2c114700 1 --2- 172.21.15.43:0/1952747771 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fef0806ace0 0x7fef0806d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fef14009b80 tx=0x7fef14008040 comp rx=0 tx=0).stop 2023-06-28T20:07:56.897 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.894+0000 7fef2c114700 1 -- 172.21.15.43:0/1952747771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef240ce110 msgr2=0x7fef240c3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:56.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.894+0000 7fef2c114700 1 --2- 172.21.15.43:0/1952747771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef240ce110 0x7fef240c3620 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fef1c001330 tx=0x7fef1c00a640 comp rx=0 tx=0).stop 2023-06-28T20:07:56.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.894+0000 7fef2c114700 1 -- 172.21.15.43:0/1952747771 shutdown_connections 2023-06-28T20:07:56.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.894+0000 7fef2c114700 1 --2- 172.21.15.43:0/1952747771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef240ce110 0x7fef240c3620 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.894+0000 7fef2c114700 1 --2- 172.21.15.43:0/1952747771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef10004430 0x7fef240b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.894+0000 7fef2c114700 1 --2- 172.21.15.43:0/1952747771 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fef0806ace0 0x7fef0806d190 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.894+0000 7fef2c114700 1 --2- 172.21.15.43:0/1952747771 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fef2414f3c0 0x7fef2415b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:56.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.894+0000 7fef2c114700 1 -- 172.21.15.43:0/1952747771 >> 172.21.15.43:0/1952747771 conn(0x7fef2406b000 msgr2=0x7fef240cfb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:56.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.894+0000 7fef2c114700 1 -- 172.21.15.43:0/1952747771 shutdown_connections 2023-06-28T20:07:56.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:56.894+0000 7fef2c114700 1 -- 172.21.15.43:0/1952747771 wait complete. 2023-06-28T20:07:56.899 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2023-06-28T20:07:56.909 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-06-28T20:07:56.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:56.911 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-06-28T20:07:56.946 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-06-28T20:07:56.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:56.947 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-06-28T20:07:57.010 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:07:57.011 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:07:57.032 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:07:57.032 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:07:57.093 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:07:57.094 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:07:57.154 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:07:57.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.276+0000 7f70b948b700 1 Processor -- start 2023-06-28T20:07:57.280 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.277+0000 7f70b948b700 1 -- start start 2023-06-28T20:07:57.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.277+0000 7f70b948b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70b415c8d0 0x7f70b40d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:57.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.277+0000 7f70b948b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f70b406a600 0x7f70b40ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:57.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.277+0000 7f70b948b700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b40d0150 con 0x7f70b40cea60 2023-06-28T20:07:57.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.277+0000 7f70b948b700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b40d0b20 con 0x7f70b406a600 2023-06-28T20:07:57.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.277+0000 7f70b948b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b40cf540 con 0x7f70b415c8d0 2023-06-28T20:07:57.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.278+0000 7f70b27fc700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f70b40cea60 0x7f70b4069ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43334/0 (socket says 172.21.15.43:43334) 2023-06-28T20:07:57.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.278+0000 7f70b27fc700 1 -- 172.21.15.43:0/1190000638 learned_addr learned my addr 172.21.15.43:0/1190000638 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:57.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.278+0000 7f70b37fe700 1 --2- 172.21.15.43:0/1190000638 >> v2:172.21.15.43:3300/0 conn(0x7f70b406a600 0x7f70b40ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:57.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.278+0000 7f70b2ffd700 1 --2- 172.21.15.43:0/1190000638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70b415c8d0 0x7f70b40d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:57.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.278+0000 7f70b37fe700 1 -- 172.21.15.43:0/1190000638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70b415c8d0 msgr2=0x7f70b40d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:57.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.278+0000 7f70b37fe700 1 --2- 172.21.15.43:0/1190000638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70b415c8d0 0x7f70b40d05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.278+0000 7f70b37fe700 1 -- 172.21.15.43:0/1190000638 >> v1:172.21.15.43:6789/0 conn(0x7f70b40cea60 legacy=0x7f70b4069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.278+0000 7f70b1ffb700 1 -- 172.21.15.43:0/1190000638 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2382932628 0 0) 0x7f70b40d0150 con 0x7f70b40cea60 2023-06-28T20:07:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.278+0000 7f70b37fe700 1 -- 172.21.15.43:0/1190000638 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b40cf820 con 0x7f70b406a600 2023-06-28T20:07:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.278+0000 7f70b2ffd700 1 --2- 172.21.15.43:0/1190000638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70b415c8d0 0x7f70b40d05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.278+0000 7f70b37fe700 1 --2- 172.21.15.43:0/1190000638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b406a600 0x7f70b40ceed0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f70ac009d90 tx=0x7f70ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b53267692c336c server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.278+0000 7f70b1ffb700 1 -- 172.21.15.43:0/1190000638 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac00e070 con 0x7f70b406a600 2023-06-28T20:07:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.278+0000 7f70b1ffb700 1 -- 172.21.15.43:0/1190000638 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ac00bbd0 con 0x7f70b406a600 2023-06-28T20:07:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.278+0000 7f70b1ffb700 1 -- 172.21.15.43:0/1190000638 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac012430 con 0x7f70b406a600 2023-06-28T20:07:57.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.279+0000 7f70b948b700 1 -- 172.21.15.43:0/1190000638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b406a600 msgr2=0x7f70b40ceed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:57.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.279+0000 7f70b948b700 1 --2- 172.21.15.43:0/1190000638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b406a600 0x7f70b40ceed0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f70ac009d90 tx=0x7f70ac00b330 comp rx=0 tx=0).stop 2023-06-28T20:07:57.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.279+0000 7f70b948b700 1 -- 172.21.15.43:0/1190000638 shutdown_connections 2023-06-28T20:07:57.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.279+0000 7f70b948b700 1 --2- 172.21.15.43:0/1190000638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b406a600 0x7f70b40ceed0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.279+0000 7f70b948b700 1 --2- 172.21.15.43:0/1190000638 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70b415c8d0 0x7f70b40d05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.279+0000 7f70b948b700 1 -- 172.21.15.43:0/1190000638 >> 172.21.15.43:0/1190000638 conn(0x7f70b406b000 msgr2=0x7f70b406b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:57.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.279+0000 7f70b948b700 1 -- 172.21.15.43:0/1190000638 shutdown_connections 2023-06-28T20:07:57.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.279+0000 7f70b948b700 1 -- 172.21.15.43:0/1190000638 wait complete. 2023-06-28T20:07:57.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.280+0000 7f70b948b700 1 Processor -- start 2023-06-28T20:07:57.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.280+0000 7f70b948b700 1 -- start start 2023-06-28T20:07:57.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.281+0000 7f70b948b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70b406a600 0x7f70b40c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:57.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.281+0000 7f70b948b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70b40cea60 0x7f70b415d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:57.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.281+0000 7f70b948b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b415c8d0 0x7f70b41f1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:57.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.281+0000 7f70b948b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b4160010 con 0x7f70b415c8d0 2023-06-28T20:07:57.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.281+0000 7f70b948b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b415fb80 con 0x7f70b40cea60 2023-06-28T20:07:57.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.281+0000 7f70b948b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b415ecb0 con 0x7f70b406a600 2023-06-28T20:07:57.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.281+0000 7f70b37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b415c8d0 0x7f70b41f1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:57.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.281+0000 7f70b37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b415c8d0 0x7f70b41f1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54158/0 (socket says 172.21.15.43:54158) 2023-06-28T20:07:57.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.281+0000 7f70b37fe700 1 -- 172.21.15.43:0/3576453444 learned_addr learned my addr 172.21.15.43:0/3576453444 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:57.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.281+0000 7f70b2ffd700 1 --2- 172.21.15.43:0/3576453444 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70b406a600 0x7f70b40c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:57.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.281+0000 7f70b27fc700 1 --2- 172.21.15.43:0/3576453444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70b40cea60 0x7f70b415d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:57.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.282+0000 7f70b37fe700 1 -- 172.21.15.43:0/3576453444 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70b406a600 msgr2=0x7f70b40c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:57.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.282+0000 7f70b37fe700 1 --2- 172.21.15.43:0/3576453444 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70b406a600 0x7f70b40c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.282+0000 7f70b37fe700 1 -- 172.21.15.43:0/3576453444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70b40cea60 msgr2=0x7f70b415d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:57.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.282+0000 7f70b37fe700 1 --2- 172.21.15.43:0/3576453444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70b40cea60 0x7f70b415d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.282+0000 7f70b37fe700 1 -- 172.21.15.43:0/3576453444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ac009a70 con 0x7f70b415c8d0 2023-06-28T20:07:57.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.282+0000 7f70b27fc700 1 --2- 172.21.15.43:0/3576453444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70b40cea60 0x7f70b415d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:57.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.282+0000 7f70b37fe700 1 --2- 172.21.15.43:0/3576453444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b415c8d0 0x7f70b41f1df0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f70ac014040 tx=0x7f70ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:57.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.282+0000 7f70ab7fe700 1 -- 172.21.15.43:0/3576453444 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac012430 con 0x7f70b415c8d0 2023-06-28T20:07:57.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.282+0000 7f70ab7fe700 1 -- 172.21.15.43:0/3576453444 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ac020070 con 0x7f70b415c8d0 2023-06-28T20:07:57.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.282+0000 7f70b948b700 1 -- 172.21.15.43:0/3576453444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b415ef70 con 0x7f70b415c8d0 2023-06-28T20:07:57.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.282+0000 7f70ab7fe700 1 -- 172.21.15.43:0/3576453444 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac004280 con 0x7f70b415c8d0 2023-06-28T20:07:57.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.282+0000 7f70b948b700 1 -- 172.21.15.43:0/3576453444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b415f530 con 0x7f70b415c8d0 2023-06-28T20:07:57.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.283+0000 7f70b948b700 1 -- 172.21.15.43:0/3576453444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b415fd30 con 0x7f70b415c8d0 2023-06-28T20:07:57.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.283+0000 7f70ab7fe700 1 -- 172.21.15.43:0/3576453444 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f70ac0043e0 con 0x7f70b415c8d0 2023-06-28T20:07:57.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.284+0000 7f70ab7fe700 1 --2- 172.21.15.43:0/3576453444 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f709406ace0 0x7f709406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:57.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.284+0000 7f70ab7fe700 1 -- 172.21.15.43:0/3576453444 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f70ac022070 con 0x7f70b415c8d0 2023-06-28T20:07:57.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.284+0000 7f70b2ffd700 1 --2- 172.21.15.43:0/3576453444 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f709406ace0 0x7f709406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:57.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.284+0000 7f70b2ffd700 1 --2- 172.21.15.43:0/3576453444 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f709406ace0 0x7f709406d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f70a4006fd0 tx=0x7f70a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:57.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.288+0000 7f70ab7fe700 1 -- 172.21.15.43:0/3576453444 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f70b415fd30 con 0x7f70b415c8d0 2023-06-28T20:07:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.488+0000 7f70b948b700 1 -- 172.21.15.43:0/3576453444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70b4072540 con 0x7f70b415c8d0 2023-06-28T20:07:57.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.489+0000 7f70ab7fe700 1 -- 172.21.15.43:0/3576453444 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18221 (secure 0 0 0) 0x7f70ac056790 con 0x7f70b415c8d0 2023-06-28T20:07:57.492 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:57.493 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:07:22.377248+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:07:18.624759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-28T20:07:19.760017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:07:57.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.491+0000 7f70b948b700 1 -- 172.21.15.43:0/3576453444 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f709406ace0 msgr2=0x7f709406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:57.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.491+0000 7f70b948b700 1 --2- 172.21.15.43:0/3576453444 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f709406ace0 0x7f709406d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f70a4006fd0 tx=0x7f70a4008040 comp rx=0 tx=0).stop 2023-06-28T20:07:57.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.491+0000 7f70b948b700 1 -- 172.21.15.43:0/3576453444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b415c8d0 msgr2=0x7f70b41f1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:57.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.491+0000 7f70b948b700 1 --2- 172.21.15.43:0/3576453444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b415c8d0 0x7f70b41f1df0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f70ac014040 tx=0x7f70ac004ad0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.491+0000 7f70b948b700 1 -- 172.21.15.43:0/3576453444 shutdown_connections 2023-06-28T20:07:57.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.491+0000 7f70b948b700 1 --2- 172.21.15.43:0/3576453444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70b415c8d0 0x7f70b41f1df0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.491+0000 7f70b948b700 1 --2- 172.21.15.43:0/3576453444 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f709406ace0 0x7f709406d190 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.491+0000 7f70b948b700 1 --2- 172.21.15.43:0/3576453444 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f70b406a600 0x7f70b40c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.491+0000 7f70b948b700 1 --2- 172.21.15.43:0/3576453444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70b40cea60 0x7f70b415d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.491+0000 7f70b948b700 1 -- 172.21.15.43:0/3576453444 >> 172.21.15.43:0/3576453444 conn(0x7f70b406b000 msgr2=0x7f70b40b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:57.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.491+0000 7f70b948b700 1 -- 172.21.15.43:0/3576453444 shutdown_connections 2023-06-28T20:07:57.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.491+0000 7f70b948b700 1 -- 172.21.15.43:0/3576453444 wait complete. 2023-06-28T20:07:57.506 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-28T20:07:13.580969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-28T20:07:14.038358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-28T20:07:18.624759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-28T20:07:19.760017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-28T20:07:57.507 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-28T20:07:57.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.632+0000 7f6264081700 1 Processor -- start 2023-06-28T20:07:57.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.633+0000 7f6264081700 1 -- start start 2023-06-28T20:07:57.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.633+0000 7f6264081700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f625c06a830 0x7f625c0c9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:57.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.633+0000 7f6264081700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f625c06ac00 0x7f625c06b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:57.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.633+0000 7f6264081700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f625c069ef0 0x7f625c06c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:57.639 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.633+0000 7f6264081700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f625c0ce370 con 0x7f625c069ef0 2023-06-28T20:07:57.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.633+0000 7f6264081700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f625c06b070 con 0x7f625c06ac00 2023-06-28T20:07:57.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.633+0000 7f6264081700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625c0ce610 con 0x7f625c06a830 2023-06-28T20:07:57.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.634+0000 7f626261e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f625c069ef0 0x7f625c06c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:57.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.634+0000 7f626261e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f625c069ef0 0x7f625c06c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54166/0 (socket says 172.21.15.43:54166) 2023-06-28T20:07:57.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.634+0000 7f626261e700 1 -- 172.21.15.43:0/1933684465 learned_addr learned my addr 172.21.15.43:0/1933684465 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:57.640 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.634+0000 7f6261e1d700 1 --2- 172.21.15.43:0/1933684465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f625c06a830 0x7f625c0c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:57.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.634+0000 7f626161c700 1 --2- 172.21.15.43:0/1933684465 >> v2:172.21.15.178:3300/0 conn(0x7f625c06ac00 0x7f625c06b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:57.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.634+0000 7f626261e700 1 -- 172.21.15.43:0/1933684465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f625c06a830 msgr2=0x7f625c0c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:57.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.634+0000 7f626261e700 1 --2- 172.21.15.43:0/1933684465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f625c06a830 0x7f625c0c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.634+0000 7f626261e700 1 -- 172.21.15.43:0/1933684465 >> v2:172.21.15.178:3300/0 conn(0x7f625c06ac00 msgr2=0x7f625c06b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:57.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.634+0000 7f626261e700 1 --2- 172.21.15.43:0/1933684465 >> v2:172.21.15.178:3300/0 conn(0x7f625c06ac00 0x7f625c06b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.634+0000 7f626261e700 1 -- 172.21.15.43:0/1933684465 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625c0ce8f0 con 0x7f625c069ef0 2023-06-28T20:07:57.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.634+0000 7f626261e700 1 --2- 172.21.15.43:0/1933684465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f625c069ef0 0x7f625c06c0a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f625c06fd80 tx=0x7f625400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4f10635a3f0b85 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:57.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.634+0000 7f6260e1b700 1 -- 172.21.15.43:0/1933684465 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625400e070 con 0x7f625c069ef0 2023-06-28T20:07:57.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.634+0000 7f6260e1b700 1 -- 172.21.15.43:0/1933684465 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625400b9c0 con 0x7f625c069ef0 2023-06-28T20:07:57.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.634+0000 7f6260e1b700 1 -- 172.21.15.43:0/1933684465 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625400bd70 con 0x7f625c069ef0 2023-06-28T20:07:57.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.635+0000 7f6264081700 1 -- 172.21.15.43:0/1933684465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f625c069ef0 msgr2=0x7f625c06c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:57.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.635+0000 7f6264081700 1 --2- 172.21.15.43:0/1933684465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f625c069ef0 0x7f625c06c0a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f625c06fd80 tx=0x7f625400b330 comp rx=0 tx=0).stop 2023-06-28T20:07:57.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.635+0000 7f6264081700 1 -- 172.21.15.43:0/1933684465 shutdown_connections 2023-06-28T20:07:57.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.635+0000 7f6264081700 1 --2- 172.21.15.43:0/1933684465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f625c069ef0 0x7f625c06c0a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.642 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.635+0000 7f6264081700 1 --2- 172.21.15.43:0/1933684465 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f625c06a830 0x7f625c0c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.635+0000 7f6264081700 1 --2- 172.21.15.43:0/1933684465 >> v2:172.21.15.178:3300/0 conn(0x7f625c06ac00 0x7f625c06b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.635+0000 7f6264081700 1 -- 172.21.15.43:0/1933684465 >> 172.21.15.43:0/1933684465 conn(0x7f625c0b1a50 msgr2=0x7f625c0cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:57.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.635+0000 7f6264081700 1 -- 172.21.15.43:0/1933684465 shutdown_connections 2023-06-28T20:07:57.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.635+0000 7f6264081700 1 -- 172.21.15.43:0/1933684465 wait complete. 2023-06-28T20:07:57.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.636+0000 7f6264081700 1 Processor -- start 2023-06-28T20:07:57.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.636+0000 7f6264081700 1 -- start start 2023-06-28T20:07:57.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.637+0000 7f6264081700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f625c069ef0 0x7f625c1653c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:57.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.637+0000 7f6264081700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f625c06a830 0x7f625c0c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:57.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.637+0000 7f6264081700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f625c06ac00 0x7f625c157d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:57.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.637+0000 7f6264081700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625c164c00 con 0x7f625c06ac00 2023-06-28T20:07:57.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.637+0000 7f6264081700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625c0c81c0 con 0x7f625c06a830 2023-06-28T20:07:57.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.637+0000 7f6264081700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625c0c84a0 con 0x7f625c069ef0 2023-06-28T20:07:57.647 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.637+0000 7f626261e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f625c06ac00 0x7f625c157d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:57.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.637+0000 7f626261e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f625c06ac00 0x7f625c157d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54172/0 (socket says 172.21.15.43:54172) 2023-06-28T20:07:57.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.637+0000 7f626261e700 1 -- 172.21.15.43:0/4054871992 learned_addr learned my addr 172.21.15.43:0/4054871992 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:57.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.637+0000 7f6261e1d700 1 --2- 172.21.15.43:0/4054871992 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f625c069ef0 0x7f625c1653c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:57.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.637+0000 7f626161c700 1 --2- 172.21.15.43:0/4054871992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f625c06a830 0x7f625c0c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:57.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.637+0000 7f626261e700 1 -- 172.21.15.43:0/4054871992 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f625c069ef0 msgr2=0x7f625c1653c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:57.648 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.637+0000 7f626261e700 1 --2- 172.21.15.43:0/4054871992 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f625c069ef0 0x7f625c1653c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.637+0000 7f626261e700 1 -- 172.21.15.43:0/4054871992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f625c06a830 msgr2=0x7f625c0c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:57.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.637+0000 7f626261e700 1 --2- 172.21.15.43:0/4054871992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f625c06a830 0x7f625c0c78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.637+0000 7f626261e700 1 -- 172.21.15.43:0/4054871992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6254009b30 con 0x7f625c06ac00 2023-06-28T20:07:57.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.637+0000 7f6261e1d700 1 --2- 172.21.15.43:0/4054871992 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f625c069ef0 0x7f625c1653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:57.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.637+0000 7f626161c700 1 --2- 172.21.15.43:0/4054871992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f625c06a830 0x7f625c0c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:57.649 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.637+0000 7f626261e700 1 --2- 172.21.15.43:0/4054871992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f625c06ac00 0x7f625c157d50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f625c06fd50 tx=0x7f6254004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:57.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.638+0000 7f62527fc700 1 -- 172.21.15.43:0/4054871992 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6254004250 con 0x7f625c06ac00 2023-06-28T20:07:57.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.638+0000 7f62527fc700 1 -- 172.21.15.43:0/4054871992 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62540043b0 con 0x7f625c06ac00 2023-06-28T20:07:57.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.638+0000 7f6264081700 1 -- 172.21.15.43:0/4054871992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f625c164ea0 con 0x7f625c06ac00 2023-06-28T20:07:57.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.638+0000 7f62527fc700 1 -- 172.21.15.43:0/4054871992 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6254004740 con 0x7f625c06ac00 2023-06-28T20:07:57.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.638+0000 7f6264081700 1 -- 172.21.15.43:0/4054871992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f625c0c8780 con 0x7f625c06ac00 2023-06-28T20:07:57.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.639+0000 7f6264081700 1 -- 172.21.15.43:0/4054871992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f625c1650e0 con 0x7f625c06ac00 2023-06-28T20:07:57.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.639+0000 7f62527fc700 1 -- 172.21.15.43:0/4054871992 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f625401b430 con 0x7f625c06ac00 2023-06-28T20:07:57.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.640+0000 7f62527fc700 1 --2- 172.21.15.43:0/4054871992 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f624006ad10 0x7f624006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:57.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.640+0000 7f62527fc700 1 -- 172.21.15.43:0/4054871992 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f625408a530 con 0x7f625c06ac00 2023-06-28T20:07:57.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.640+0000 7f6261e1d700 1 --2- 172.21.15.43:0/4054871992 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f624006ad10 0x7f624006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:57.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.640+0000 7f6261e1d700 1 --2- 172.21.15.43:0/4054871992 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f624006ad10 0x7f624006d1c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f624c0099e0 tx=0x7f624c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:57.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.644+0000 7f62527fc700 1 -- 172.21.15.43:0/4054871992 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f625c1650e0 con 0x7f625c06ac00 2023-06-28T20:07:57.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.842+0000 7f6264081700 1 -- 172.21.15.43:0/4054871992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f625c1650e0 con 0x7f625c06ac00 2023-06-28T20:07:57.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.843+0000 7f62527fc700 1 -- 172.21.15.43:0/4054871992 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f625c1650e0 con 0x7f625c06ac00 2023-06-28T20:07:57.847 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-06-28T20:07:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.844+0000 7f6264081700 1 -- 172.21.15.43:0/4054871992 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f624006ad10 msgr2=0x7f624006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:57.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.844+0000 7f6264081700 1 --2- 172.21.15.43:0/4054871992 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f624006ad10 0x7f624006d1c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f624c0099e0 tx=0x7f624c008040 comp rx=0 tx=0).stop 2023-06-28T20:07:57.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.844+0000 7f6264081700 1 -- 172.21.15.43:0/4054871992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f625c06ac00 msgr2=0x7f625c157d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:57.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.844+0000 7f6264081700 1 --2- 172.21.15.43:0/4054871992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f625c06ac00 0x7f625c157d50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f625c06fd50 tx=0x7f6254004ad0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.845+0000 7f6264081700 1 -- 172.21.15.43:0/4054871992 shutdown_connections 2023-06-28T20:07:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.845+0000 7f6264081700 1 --2- 172.21.15.43:0/4054871992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f625c06ac00 0x7f625c157d50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.845+0000 7f6264081700 1 --2- 172.21.15.43:0/4054871992 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f624006ad10 0x7f624006d1c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.845+0000 7f6264081700 1 --2- 172.21.15.43:0/4054871992 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f625c069ef0 0x7f625c1653c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.845+0000 7f6264081700 1 --2- 172.21.15.43:0/4054871992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f625c06a830 0x7f625c0c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.845+0000 7f6264081700 1 -- 172.21.15.43:0/4054871992 >> 172.21.15.43:0/4054871992 conn(0x7f625c0b1a50 msgr2=0x7f625c0b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.845+0000 7f6264081700 1 -- 172.21.15.43:0/4054871992 shutdown_connections 2023-06-28T20:07:57.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.845+0000 7f6264081700 1 -- 172.21.15.43:0/4054871992 wait complete. 2023-06-28T20:07:57.860 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-28T20:07:57.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.985+0000 7f85e6937700 1 Processor -- start 2023-06-28T20:07:57.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.985+0000 7f85e6937700 1 -- start start 2023-06-28T20:07:57.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.986+0000 7f85e6937700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f85e00b1f90 0x7f85e0160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:57.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.986+0000 7f85e6937700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e0160250 con 0x7f85e00af060 2023-06-28T20:07:57.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.986+0000 7f85e6937700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85e0071410 con 0x7f85e00aec90 2023-06-28T20:07:57.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.986+0000 7f85e6937700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e015cb30 con 0x7f85e00b1f90 2023-06-28T20:07:57.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.986+0000 7f85df7fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f85e00af060 0x7f85e00b1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43362/0 (socket says 172.21.15.43:43362) 2023-06-28T20:07:57.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.986+0000 7f85df7fe700 1 -- 172.21.15.43:0/1770759567 learned_addr learned my addr 172.21.15.43:0/1770759567 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:57.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.986+0000 7f85e4ed4700 1 --2- 172.21.15.43:0/1770759567 >> v2:172.21.15.178:3300/0 conn(0x7f85e00b1f90 0x7f85e0160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:57.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.986+0000 7f85deffd700 1 -- 172.21.15.43:0/1770759567 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 421286211 0 0) 0x7f85e0160250 con 0x7f85e00af060 2023-06-28T20:07:57.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.986+0000 7f85deffd700 1 -- 172.21.15.43:0/1770759567 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85c80035b0 con 0x7f85e00af060 2023-06-28T20:07:57.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.987+0000 7f85deffd700 1 -- 172.21.15.43:0/1770759567 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 705290039 0 0) 0x7f85c80035b0 con 0x7f85e00af060 2023-06-28T20:07:57.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.987+0000 7f85deffd700 1 -- 172.21.15.43:0/1770759567 >> v2:172.21.15.178:3300/0 conn(0x7f85e00b1f90 msgr2=0x7f85e0160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:57.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.987+0000 7f85deffd700 1 --2- 172.21.15.43:0/1770759567 >> v2:172.21.15.178:3300/0 conn(0x7f85e00b1f90 0x7f85e0160cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.987+0000 7f85deffd700 1 -- 172.21.15.43:0/1770759567 >> v1:172.21.15.178:6789/0 conn(0x7f85e00aec90 legacy=0x7f85e0070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:57.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.987+0000 7f85deffd700 1 -- 172.21.15.43:0/1770759567 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e01604f0 con 0x7f85e00af060 2023-06-28T20:07:57.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.987+0000 7f85deffd700 1 -- 172.21.15.43:0/1770759567 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f85d4001670 con 0x7f85e00af060 2023-06-28T20:07:57.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.987+0000 7f85deffd700 1 -- 172.21.15.43:0/1770759567 >> v1:172.21.15.43:6789/0 conn(0x7f85e00af060 legacy=0x7f85e00b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:57.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.987+0000 7f85deffd700 1 --2- 172.21.15.43:0/1770759567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85c8003a00 0x7f85c8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:57.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.987+0000 7f85deffd700 1 --2- 172.21.15.43:0/1770759567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85c8004850 0x7f85c8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:57.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.987+0000 7f85deffd700 1 --2- 172.21.15.43:0/1770759567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c8005200 0x7f85c800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:57.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.987+0000 7f85deffd700 1 -- 172.21.15.43:0/1770759567 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e015cb30 con 0x7f85c8005200 2023-06-28T20:07:57.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.987+0000 7f85deffd700 1 -- 172.21.15.43:0/1770759567 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c800c080 con 0x7f85c8004850 2023-06-28T20:07:57.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.987+0000 7f85deffd700 1 -- 172.21.15.43:0/1770759567 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c800c240 con 0x7f85c8003a00 2023-06-28T20:07:57.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.987+0000 7f85e4ed4700 1 --2- 172.21.15.43:0/1770759567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c8005200 0x7f85c800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:57.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.987+0000 7f85df7fe700 1 --2- 172.21.15.43:0/1770759567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85c8004850 0x7f85c8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:57.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.987+0000 7f85dffff700 1 --2- 172.21.15.43:0/1770759567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85c8003a00 0x7f85c8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:57.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.987+0000 7f85e4ed4700 1 -- 172.21.15.43:0/1770759567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85c8003a00 msgr2=0x7f85c8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:57.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.987+0000 7f85e4ed4700 1 --2- 172.21.15.43:0/1770759567 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85c8003a00 0x7f85c8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.987+0000 7f85e4ed4700 1 -- 172.21.15.43:0/1770759567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85c8004850 msgr2=0x7f85c8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:57.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.987+0000 7f85e4ed4700 1 --2- 172.21.15.43:0/1770759567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85c8004850 0x7f85c8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.987+0000 7f85e4ed4700 1 -- 172.21.15.43:0/1770759567 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c80035b0 con 0x7f85c8005200 2023-06-28T20:07:57.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.987+0000 7f85e4ed4700 1 -- 172.21.15.43:0/1770759567 reap_dead start 2023-06-28T20:07:57.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.987+0000 7f85e4ed4700 1 --2- 172.21.15.43:0/1770759567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c8005200 0x7f85c800ba10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009f90 tx=0x7f85c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402f4432bd90d688 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:57.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.988+0000 7f85deffd700 1 -- 172.21.15.43:0/1770759567 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d8003a20 con 0x7f85c8005200 2023-06-28T20:07:57.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.988+0000 7f85deffd700 1 -- 172.21.15.43:0/1770759567 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d800f040 con 0x7f85c8005200 2023-06-28T20:07:57.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.988+0000 7f85deffd700 1 -- 172.21.15.43:0/1770759567 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d8003da0 con 0x7f85c8005200 2023-06-28T20:07:57.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.988+0000 7f85e6937700 1 -- 172.21.15.43:0/1770759567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c8005200 msgr2=0x7f85c800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:57.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.988+0000 7f85e6937700 1 --2- 172.21.15.43:0/1770759567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c8005200 0x7f85c800ba10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009f90 tx=0x7f85c8005880 comp rx=0 tx=0).stop 2023-06-28T20:07:57.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.988+0000 7f85e6937700 1 -- 172.21.15.43:0/1770759567 shutdown_connections 2023-06-28T20:07:57.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.988+0000 7f85e6937700 1 --2- 172.21.15.43:0/1770759567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c8005200 0x7f85c800ba10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:57.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.988+0000 7f85e6937700 1 -- 172.21.15.43:0/1770759567 >> 172.21.15.43:0/1770759567 conn(0x7f85e006b000 msgr2=0x7f85e006b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:57.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.988+0000 7f85e6937700 1 -- 172.21.15.43:0/1770759567 shutdown_connections 2023-06-28T20:07:57.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.988+0000 7f85e6937700 1 -- 172.21.15.43:0/1770759567 wait complete. 2023-06-28T20:07:57.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.989+0000 7f85e6937700 1 Processor -- start 2023-06-28T20:07:57.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.990+0000 7f85e6937700 1 -- start start 2023-06-28T20:07:58.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.990+0000 7f85e6937700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c8005200 0x7f85e006a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:58.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.990+0000 7f85e6937700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85e00b2940 0x7f85e00b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:58.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.990+0000 7f85e6937700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85e01e5fb0 0x7f85e01e6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:58.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.990+0000 7f85e6937700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e00b3630 con 0x7f85c8005200 2023-06-28T20:07:58.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.990+0000 7f85e6937700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e00b3910 con 0x7f85e00b2940 2023-06-28T20:07:58.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.990+0000 7f85e6937700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e00c7260 con 0x7f85e01e5fb0 2023-06-28T20:07:58.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.990+0000 7f85dffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c8005200 0x7f85e006a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:58.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.990+0000 7f85dffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c8005200 0x7f85e006a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54188/0 (socket says 172.21.15.43:54188) 2023-06-28T20:07:58.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.990+0000 7f85dffff700 1 -- 172.21.15.43:0/4002341020 learned_addr learned my addr 172.21.15.43:0/4002341020 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:58.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.991+0000 7f85df7fe700 1 --2- 172.21.15.43:0/4002341020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85e00b2940 0x7f85e00b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:58.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.991+0000 7f85e4ed4700 1 --2- 172.21.15.43:0/4002341020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85e01e5fb0 0x7f85e01e6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:58.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.991+0000 7f85dffff700 1 -- 172.21.15.43:0/4002341020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85e01e5fb0 msgr2=0x7f85e01e6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:58.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.991+0000 7f85dffff700 1 --2- 172.21.15.43:0/4002341020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85e01e5fb0 0x7f85e01e6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.991+0000 7f85dffff700 1 -- 172.21.15.43:0/4002341020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85e00b2940 msgr2=0x7f85e00b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:58.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.991+0000 7f85dffff700 1 --2- 172.21.15.43:0/4002341020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85e00b2940 0x7f85e00b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.991+0000 7f85dffff700 1 -- 172.21.15.43:0/4002341020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e00c7540 con 0x7f85c8005200 2023-06-28T20:07:58.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.991+0000 7f85e4ed4700 1 --2- 172.21.15.43:0/4002341020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85e01e5fb0 0x7f85e01e6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:58.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.991+0000 7f85df7fe700 1 --2- 172.21.15.43:0/4002341020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85e00b2940 0x7f85e00b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:58.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.991+0000 7f85dffff700 1 --2- 172.21.15.43:0/4002341020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c8005200 0x7f85e006a1c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f85d000f820 tx=0x7f85d000fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:58.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.991+0000 7f85dcff9700 1 -- 172.21.15.43:0/4002341020 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d00052e0 con 0x7f85c8005200 2023-06-28T20:07:58.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.991+0000 7f85dcff9700 1 -- 172.21.15.43:0/4002341020 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d0005460 con 0x7f85c8005200 2023-06-28T20:07:58.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.991+0000 7f85e6937700 1 -- 172.21.15.43:0/4002341020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e00c7820 con 0x7f85c8005200 2023-06-28T20:07:58.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.991+0000 7f85dcff9700 1 -- 172.21.15.43:0/4002341020 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d00038c0 con 0x7f85c8005200 2023-06-28T20:07:58.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.991+0000 7f85e6937700 1 -- 172.21.15.43:0/4002341020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e006a700 con 0x7f85c8005200 2023-06-28T20:07:58.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.992+0000 7f85e6937700 1 -- 172.21.15.43:0/4002341020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e00c7a40 con 0x7f85c8005200 2023-06-28T20:07:58.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.992+0000 7f85dcff9700 1 -- 172.21.15.43:0/4002341020 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f85d0003a20 con 0x7f85c8005200 2023-06-28T20:07:58.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.993+0000 7f85dcff9700 1 --2- 172.21.15.43:0/4002341020 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f85cc06ace0 0x7f85cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:58.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.993+0000 7f85dcff9700 1 -- 172.21.15.43:0/4002341020 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f85d0089270 con 0x7f85c8005200 2023-06-28T20:07:58.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.993+0000 7f85df7fe700 1 --2- 172.21.15.43:0/4002341020 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f85cc06ace0 0x7f85cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:58.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.993+0000 7f85df7fe700 1 --2- 172.21.15.43:0/4002341020 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f85cc06ace0 0x7f85cc06d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f85e01e7c00 tx=0x7f85d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:58.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:57.997+0000 7f85dcff9700 1 -- 172.21.15.43:0/4002341020 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f85d0054320 con 0x7f85c8005200 2023-06-28T20:07:58.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.196+0000 7f85e6937700 1 -- 172.21.15.43:0/4002341020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f85e006cde0 con 0x7f85c8005200 2023-06-28T20:07:58.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.196+0000 7f85dcff9700 1 -- 172.21.15.43:0/4002341020 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f85d0057940 con 0x7f85c8005200 2023-06-28T20:07:58.200 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-06-28T20:07:58.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.198+0000 7f85e6937700 1 -- 172.21.15.43:0/4002341020 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f85cc06ace0 msgr2=0x7f85cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:58.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.198+0000 7f85e6937700 1 --2- 172.21.15.43:0/4002341020 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f85cc06ace0 0x7f85cc06d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f85e01e7c00 tx=0x7f85d4009040 comp rx=0 tx=0).stop 2023-06-28T20:07:58.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.198+0000 7f85e6937700 1 -- 172.21.15.43:0/4002341020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c8005200 msgr2=0x7f85e006a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:58.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.198+0000 7f85e6937700 1 --2- 172.21.15.43:0/4002341020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c8005200 0x7f85e006a1c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f85d000f820 tx=0x7f85d000fbe0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.198+0000 7f85e6937700 1 -- 172.21.15.43:0/4002341020 shutdown_connections 2023-06-28T20:07:58.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.198+0000 7f85e6937700 1 --2- 172.21.15.43:0/4002341020 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f85cc06ace0 0x7f85cc06d190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.198+0000 7f85e6937700 1 --2- 172.21.15.43:0/4002341020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85e01e5fb0 0x7f85e01e6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.198+0000 7f85e6937700 1 --2- 172.21.15.43:0/4002341020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85c8005200 0x7f85e006a1c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.198+0000 7f85e6937700 1 --2- 172.21.15.43:0/4002341020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85e00b2940 0x7f85e00b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.198+0000 7f85e6937700 1 -- 172.21.15.43:0/4002341020 >> 172.21.15.43:0/4002341020 conn(0x7f85e006b000 msgr2=0x7f85e00ce590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:58.203 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.199+0000 7f85e6937700 1 -- 172.21.15.43:0/4002341020 shutdown_connections 2023-06-28T20:07:58.204 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.199+0000 7f85e6937700 1 -- 172.21.15.43:0/4002341020 wait complete. 2023-06-28T20:07:58.214 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-28T20:07:58.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.340+0000 7fa8687bd700 1 Processor -- start 2023-06-28T20:07:58.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.340+0000 7fa8687bd700 1 -- start start 2023-06-28T20:07:58.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.341+0000 7fa8687bd700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa86015cae0 0x7fa8600cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:58.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.341+0000 7fa8687bd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa8600cfe00 0x7fa86006cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:58.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.341+0000 7fa8687bd700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8600d0680 con 0x7fa86006d310 2023-06-28T20:07:58.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.341+0000 7fa8687bd700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa86006bb20 con 0x7fa8600cfe00 2023-06-28T20:07:58.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.341+0000 7fa8687bd700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa86006be00 con 0x7fa86015cae0 2023-06-28T20:07:58.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.341+0000 7fa866d5a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa8600cfe00 0x7fa86006cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:58.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.341+0000 7fa866d5a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa8600cfe00 0x7fa86006cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54196/0 (socket says 172.21.15.43:54196) 2023-06-28T20:07:58.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.341+0000 7fa866d5a700 1 -- 172.21.15.43:0/1658422454 learned_addr learned my addr 172.21.15.43:0/1658422454 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:58.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.341+0000 7fa865d58700 1 --2- 172.21.15.43:0/1658422454 >> v2:172.21.15.178:3300/0 conn(0x7fa86015cae0 0x7fa8600cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:58.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.341+0000 7fa866d5a700 1 -- 172.21.15.43:0/1658422454 >> v2:172.21.15.178:3300/0 conn(0x7fa86015cae0 msgr2=0x7fa8600cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:58.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.341+0000 7fa866d5a700 1 --2- 172.21.15.43:0/1658422454 >> v2:172.21.15.178:3300/0 conn(0x7fa86015cae0 0x7fa8600cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.341+0000 7fa866d5a700 1 -- 172.21.15.43:0/1658422454 >> v1:172.21.15.178:6789/0 conn(0x7fa86006d310 legacy=0x7fa86006d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:58.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.341+0000 7fa866d5a700 1 -- 172.21.15.43:0/1658422454 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86006c0e0 con 0x7fa8600cfe00 2023-06-28T20:07:58.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.342+0000 7fa866d5a700 1 --2- 172.21.15.43:0/1658422454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8600cfe00 0x7fa86006cbc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa8600b37c0 tx=0x7fa85800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a8bfaa66df1376 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:58.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.342+0000 7fa865557700 1 -- 172.21.15.43:0/1658422454 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85800e070 con 0x7fa8600cfe00 2023-06-28T20:07:58.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.342+0000 7fa865557700 1 -- 172.21.15.43:0/1658422454 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85800b9c0 con 0x7fa8600cfe00 2023-06-28T20:07:58.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.342+0000 7fa865557700 1 -- 172.21.15.43:0/1658422454 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85800bd70 con 0x7fa8600cfe00 2023-06-28T20:07:58.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.342+0000 7fa8687bd700 1 -- 172.21.15.43:0/1658422454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8600cfe00 msgr2=0x7fa86006cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:58.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.342+0000 7fa8687bd700 1 --2- 172.21.15.43:0/1658422454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8600cfe00 0x7fa86006cbc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa8600b37c0 tx=0x7fa85800b330 comp rx=0 tx=0).stop 2023-06-28T20:07:58.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.342+0000 7fa8687bd700 1 -- 172.21.15.43:0/1658422454 shutdown_connections 2023-06-28T20:07:58.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.342+0000 7fa8687bd700 1 --2- 172.21.15.43:0/1658422454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8600cfe00 0x7fa86006cbc0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.342+0000 7fa8687bd700 1 --2- 172.21.15.43:0/1658422454 >> v2:172.21.15.178:3300/0 conn(0x7fa86015cae0 0x7fa8600cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.342+0000 7fa8687bd700 1 -- 172.21.15.43:0/1658422454 >> 172.21.15.43:0/1658422454 conn(0x7fa8600664b0 msgr2=0x7fa8600d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:58.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.342+0000 7fa8687bd700 1 -- 172.21.15.43:0/1658422454 shutdown_connections 2023-06-28T20:07:58.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.342+0000 7fa8687bd700 1 -- 172.21.15.43:0/1658422454 wait complete. 2023-06-28T20:07:58.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.344+0000 7fa8687bd700 1 Processor -- start 2023-06-28T20:07:58.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.344+0000 7fa8687bd700 1 -- start start 2023-06-28T20:07:58.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.344+0000 7fa8687bd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa86006d310 0x7fa8600b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:58.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.344+0000 7fa8687bd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8600cfe00 0x7fa8600b1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:58.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.344+0000 7fa8687bd700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa86015cae0 0x7fa8600c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:58.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.344+0000 7fa8687bd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8601594b0 con 0x7fa8600cfe00 2023-06-28T20:07:58.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.344+0000 7fa8687bd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa860159790 con 0x7fa86006d310 2023-06-28T20:07:58.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.344+0000 7fa8687bd700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa860159a70 con 0x7fa86015cae0 2023-06-28T20:07:58.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.344+0000 7fa865d58700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8600cfe00 0x7fa8600b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:58.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.344+0000 7fa865d58700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8600cfe00 0x7fa8600b1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54204/0 (socket says 172.21.15.43:54204) 2023-06-28T20:07:58.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.345+0000 7fa865d58700 1 -- 172.21.15.43:0/2531747996 learned_addr learned my addr 172.21.15.43:0/2531747996 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:58.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.345+0000 7fa866559700 1 --2- 172.21.15.43:0/2531747996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa86006d310 0x7fa8600b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:58.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.345+0000 7fa866d5a700 1 --2- 172.21.15.43:0/2531747996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa86015cae0 0x7fa8600c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:58.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.345+0000 7fa865d58700 1 -- 172.21.15.43:0/2531747996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa86015cae0 msgr2=0x7fa8600c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:58.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.345+0000 7fa865d58700 1 --2- 172.21.15.43:0/2531747996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa86015cae0 0x7fa8600c8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.345+0000 7fa865d58700 1 -- 172.21.15.43:0/2531747996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa86006d310 msgr2=0x7fa8600b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:58.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.345+0000 7fa865d58700 1 --2- 172.21.15.43:0/2531747996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa86006d310 0x7fa8600b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.345+0000 7fa865d58700 1 -- 172.21.15.43:0/2531747996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa858009b30 con 0x7fa8600cfe00 2023-06-28T20:07:58.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.345+0000 7fa865d58700 1 --2- 172.21.15.43:0/2531747996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8600cfe00 0x7fa8600b1190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00d800 tx=0x7fa85c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:58.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.345+0000 7fa856ffd700 1 -- 172.21.15.43:0/2531747996 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c0125f0 con 0x7fa8600cfe00 2023-06-28T20:07:58.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.345+0000 7fa856ffd700 1 -- 172.21.15.43:0/2531747996 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85c012750 con 0x7fa8600cfe00 2023-06-28T20:07:58.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.345+0000 7fa8687bd700 1 -- 172.21.15.43:0/2531747996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa860159d50 con 0x7fa8600cfe00 2023-06-28T20:07:58.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.345+0000 7fa856ffd700 1 -- 172.21.15.43:0/2531747996 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c012b00 con 0x7fa8600cfe00 2023-06-28T20:07:58.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.345+0000 7fa8687bd700 1 -- 172.21.15.43:0/2531747996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa860159140 con 0x7fa8600cfe00 2023-06-28T20:07:58.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.346+0000 7fa8687bd700 1 -- 172.21.15.43:0/2531747996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8600728b0 con 0x7fa8600cfe00 2023-06-28T20:07:58.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.347+0000 7fa856ffd700 1 -- 172.21.15.43:0/2531747996 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa85c012c60 con 0x7fa8600cfe00 2023-06-28T20:07:58.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.347+0000 7fa856ffd700 1 --2- 172.21.15.43:0/2531747996 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa84406ad10 0x7fa84406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:58.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.347+0000 7fa856ffd700 1 -- 172.21.15.43:0/2531747996 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa85c089410 con 0x7fa8600cfe00 2023-06-28T20:07:58.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.347+0000 7fa866559700 1 --2- 172.21.15.43:0/2531747996 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa84406ad10 0x7fa84406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:58.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.348+0000 7fa866559700 1 --2- 172.21.15.43:0/2531747996 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa84406ad10 0x7fa84406d1c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa850000fb0 tx=0x7fa850009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:58.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.352+0000 7fa856ffd700 1 -- 172.21.15.43:0/2531747996 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa85c0544c0 con 0x7fa8600cfe00 2023-06-28T20:07:58.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.549+0000 7fa8687bd700 1 -- 172.21.15.43:0/2531747996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa8600ae9b0 con 0x7fa8600cfe00 2023-06-28T20:07:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.549+0000 7fa856ffd700 1 -- 172.21.15.43:0/2531747996 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa85c057ae0 con 0x7fa8600cfe00 2023-06-28T20:07:58.553 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-06-28T20:07:58.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.551+0000 7fa8687bd700 1 -- 172.21.15.43:0/2531747996 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa84406ad10 msgr2=0x7fa84406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:58.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.551+0000 7fa8687bd700 1 --2- 172.21.15.43:0/2531747996 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa84406ad10 0x7fa84406d1c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa850000fb0 tx=0x7fa850009040 comp rx=0 tx=0).stop 2023-06-28T20:07:58.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.551+0000 7fa8687bd700 1 -- 172.21.15.43:0/2531747996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8600cfe00 msgr2=0x7fa8600b1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:58.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.551+0000 7fa8687bd700 1 --2- 172.21.15.43:0/2531747996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8600cfe00 0x7fa8600b1190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00d800 tx=0x7fa85c00db10 comp rx=0 tx=0).stop 2023-06-28T20:07:58.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.551+0000 7fa8687bd700 1 -- 172.21.15.43:0/2531747996 shutdown_connections 2023-06-28T20:07:58.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.551+0000 7fa8687bd700 1 --2- 172.21.15.43:0/2531747996 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa84406ad10 0x7fa84406d1c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.551+0000 7fa8687bd700 1 --2- 172.21.15.43:0/2531747996 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa86015cae0 0x7fa8600c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.551+0000 7fa8687bd700 1 --2- 172.21.15.43:0/2531747996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa86006d310 0x7fa8600b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.551+0000 7fa8687bd700 1 --2- 172.21.15.43:0/2531747996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8600cfe00 0x7fa8600b1190 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.551+0000 7fa8687bd700 1 -- 172.21.15.43:0/2531747996 >> 172.21.15.43:0/2531747996 conn(0x7fa8600664b0 msgr2=0x7fa86015c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:58.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.551+0000 7fa8687bd700 1 -- 172.21.15.43:0/2531747996 shutdown_connections 2023-06-28T20:07:58.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.551+0000 7fa8687bd700 1 -- 172.21.15.43:0/2531747996 wait complete. 2023-06-28T20:07:58.566 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-28T20:07:58.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.691+0000 7f2a6a157700 1 Processor -- start 2023-06-28T20:07:58.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.691+0000 7f2a6a157700 1 -- start start 2023-06-28T20:07:58.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.692+0000 7f2a6a157700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2a640aec90 0x7f2a64160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:58.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.692+0000 7f2a6a157700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a64160250 con 0x7f2a640af060 2023-06-28T20:07:58.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.692+0000 7f2a6a157700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a64071410 con 0x7f2a6406ccc0 2023-06-28T20:07:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.692+0000 7f2a6a157700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a6415cb30 con 0x7f2a640aec90 2023-06-28T20:07:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.692+0000 7f2a62ffd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2a640af060 0x7f2a64070bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43398/0 (socket says 172.21.15.43:43398) 2023-06-28T20:07:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.692+0000 7f2a62ffd700 1 -- 172.21.15.43:0/754311340 learned_addr learned my addr 172.21.15.43:0/754311340 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.692+0000 7f2a637fe700 1 --2- 172.21.15.43:0/754311340 >> v2:172.21.15.178:3300/0 conn(0x7f2a640aec90 0x7f2a64160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.692+0000 7f2a627fc700 1 -- 172.21.15.43:0/754311340 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3343368422 0 0) 0x7f2a64160250 con 0x7f2a640af060 2023-06-28T20:07:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.692+0000 7f2a627fc700 1 -- 172.21.15.43:0/754311340 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a500035b0 con 0x7f2a640af060 2023-06-28T20:07:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.692+0000 7f2a627fc700 1 -- 172.21.15.43:0/754311340 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1001702959 0 0) 0x7f2a500035b0 con 0x7f2a640af060 2023-06-28T20:07:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.692+0000 7f2a627fc700 1 -- 172.21.15.43:0/754311340 >> v2:172.21.15.178:3300/0 conn(0x7f2a640aec90 msgr2=0x7f2a64160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.692+0000 7f2a627fc700 1 --2- 172.21.15.43:0/754311340 >> v2:172.21.15.178:3300/0 conn(0x7f2a640aec90 0x7f2a64160cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.692+0000 7f2a627fc700 1 -- 172.21.15.43:0/754311340 >> v1:172.21.15.178:6789/0 conn(0x7f2a6406ccc0 legacy=0x7f2a640b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.692+0000 7f2a627fc700 1 -- 172.21.15.43:0/754311340 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a641604f0 con 0x7f2a640af060 2023-06-28T20:07:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.692+0000 7f2a627fc700 1 -- 172.21.15.43:0/754311340 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f2a4c001670 con 0x7f2a640af060 2023-06-28T20:07:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.693+0000 7f2a627fc700 1 -- 172.21.15.43:0/754311340 >> v1:172.21.15.43:6789/0 conn(0x7f2a640af060 legacy=0x7f2a64070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.693+0000 7f2a627fc700 1 --2- 172.21.15.43:0/754311340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a50003a00 0x7f2a50003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:58.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.693+0000 7f2a627fc700 1 --2- 172.21.15.43:0/754311340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a50004850 0x7f2a50004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:58.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.693+0000 7f2a627fc700 1 --2- 172.21.15.43:0/754311340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a50005200 0x7f2a5000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:58.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.693+0000 7f2a627fc700 1 -- 172.21.15.43:0/754311340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6415cb30 con 0x7f2a50005200 2023-06-28T20:07:58.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.693+0000 7f2a627fc700 1 -- 172.21.15.43:0/754311340 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5000c080 con 0x7f2a50004850 2023-06-28T20:07:58.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.693+0000 7f2a627fc700 1 -- 172.21.15.43:0/754311340 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5000c240 con 0x7f2a50003a00 2023-06-28T20:07:58.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.693+0000 7f2a62ffd700 1 --2- 172.21.15.43:0/754311340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a50004850 0x7f2a50004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:58.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.693+0000 7f2a63fff700 1 --2- 172.21.15.43:0/754311340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a50005200 0x7f2a5000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:58.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.693+0000 7f2a637fe700 1 --2- 172.21.15.43:0/754311340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a50003a00 0x7f2a50003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:58.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.693+0000 7f2a62ffd700 1 -- 172.21.15.43:0/754311340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a50003a00 msgr2=0x7f2a50003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:58.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.693+0000 7f2a62ffd700 1 --2- 172.21.15.43:0/754311340 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a50003a00 0x7f2a50003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.693+0000 7f2a62ffd700 1 -- 172.21.15.43:0/754311340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a50005200 msgr2=0x7f2a5000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:58.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.693+0000 7f2a62ffd700 1 --2- 172.21.15.43:0/754311340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a50005200 0x7f2a5000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.693+0000 7f2a62ffd700 1 -- 172.21.15.43:0/754311340 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a500035b0 con 0x7f2a50004850 2023-06-28T20:07:58.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.693+0000 7f2a63fff700 1 --2- 172.21.15.43:0/754311340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a50005200 0x7f2a5000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:07:58.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.693+0000 7f2a63fff700 1 -- 172.21.15.43:0/754311340 reap_dead start 2023-06-28T20:07:58.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.693+0000 7f2a62ffd700 1 --2- 172.21.15.43:0/754311340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a50004850 0x7f2a50004cc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c00c2a0 tx=0x7f2a50005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a42523c874324d server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:58.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.693+0000 7f2a627fc700 1 -- 172.21.15.43:0/754311340 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c007c10 con 0x7f2a50004850 2023-06-28T20:07:58.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.693+0000 7f2a627fc700 1 -- 172.21.15.43:0/754311340 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4c00cdc0 con 0x7f2a50004850 2023-06-28T20:07:58.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.693+0000 7f2a627fc700 1 -- 172.21.15.43:0/754311340 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c005ba0 con 0x7f2a50004850 2023-06-28T20:07:58.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.694+0000 7f2a6a157700 1 -- 172.21.15.43:0/754311340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a50004850 msgr2=0x7f2a50004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:58.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.694+0000 7f2a6a157700 1 --2- 172.21.15.43:0/754311340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a50004850 0x7f2a50004cc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c00c2a0 tx=0x7f2a50005880 comp rx=0 tx=0).stop 2023-06-28T20:07:58.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.694+0000 7f2a6a157700 1 -- 172.21.15.43:0/754311340 shutdown_connections 2023-06-28T20:07:58.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.694+0000 7f2a6a157700 1 --2- 172.21.15.43:0/754311340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a50004850 0x7f2a50004cc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.694+0000 7f2a6a157700 1 -- 172.21.15.43:0/754311340 >> 172.21.15.43:0/754311340 conn(0x7f2a6406b000 msgr2=0x7f2a6406b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:58.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.694+0000 7f2a6a157700 1 -- 172.21.15.43:0/754311340 shutdown_connections 2023-06-28T20:07:58.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.694+0000 7f2a6a157700 1 -- 172.21.15.43:0/754311340 wait complete. 2023-06-28T20:07:58.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.695+0000 7f2a6a157700 1 Processor -- start 2023-06-28T20:07:58.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.695+0000 7f2a6a157700 1 -- start start 2023-06-28T20:07:58.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.696+0000 7f2a6a157700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a50004850 0x7f2a640b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:58.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.696+0000 7f2a6a157700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a641f2f80 0x7f2a640ad140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:58.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.696+0000 7f2a6a157700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a64150d10 0x7f2a64151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:58.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.696+0000 7f2a6a157700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a640b2e80 con 0x7f2a641f2f80 2023-06-28T20:07:58.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.696+0000 7f2a6a157700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a641f2400 con 0x7f2a50004850 2023-06-28T20:07:58.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.696+0000 7f2a6a157700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a641f26e0 con 0x7f2a64150d10 2023-06-28T20:07:58.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.696+0000 7f2a62ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a641f2f80 0x7f2a640ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:58.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.696+0000 7f2a62ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a641f2f80 0x7f2a640ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54224/0 (socket says 172.21.15.43:54224) 2023-06-28T20:07:58.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.696+0000 7f2a62ffd700 1 -- 172.21.15.43:0/3403453661 learned_addr learned my addr 172.21.15.43:0/3403453661 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:58.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.697+0000 7f2a637fe700 1 --2- 172.21.15.43:0/3403453661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a50004850 0x7f2a640b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:58.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.697+0000 7f2a63fff700 1 --2- 172.21.15.43:0/3403453661 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a64150d10 0x7f2a64151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:58.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.697+0000 7f2a62ffd700 1 -- 172.21.15.43:0/3403453661 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a64150d10 msgr2=0x7f2a64151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:58.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.697+0000 7f2a62ffd700 1 --2- 172.21.15.43:0/3403453661 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a64150d10 0x7f2a64151180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.697+0000 7f2a62ffd700 1 -- 172.21.15.43:0/3403453661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a50004850 msgr2=0x7f2a640b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:58.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.697+0000 7f2a62ffd700 1 --2- 172.21.15.43:0/3403453661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a50004850 0x7f2a640b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.697+0000 7f2a62ffd700 1 -- 172.21.15.43:0/3403453661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a641f29c0 con 0x7f2a641f2f80 2023-06-28T20:07:58.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.697+0000 7f2a63fff700 1 --2- 172.21.15.43:0/3403453661 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a64150d10 0x7f2a64151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:58.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.697+0000 7f2a62ffd700 1 --2- 172.21.15.43:0/3403453661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a641f2f80 0x7f2a640ad140 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2a5000c210 tx=0x7f2a641603e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:58.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.697+0000 7f2a5bfff700 1 -- 172.21.15.43:0/3403453661 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c001640 con 0x7f2a641f2f80 2023-06-28T20:07:58.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.697+0000 7f2a5bfff700 1 -- 172.21.15.43:0/3403453661 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4c0070d0 con 0x7f2a641f2f80 2023-06-28T20:07:58.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.697+0000 7f2a6a157700 1 -- 172.21.15.43:0/3403453661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a641f2ca0 con 0x7f2a641f2f80 2023-06-28T20:07:58.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.697+0000 7f2a5bfff700 1 -- 172.21.15.43:0/3403453661 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c007440 con 0x7f2a641f2f80 2023-06-28T20:07:58.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.697+0000 7f2a6a157700 1 -- 172.21.15.43:0/3403453661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a641516c0 con 0x7f2a641f2f80 2023-06-28T20:07:58.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.698+0000 7f2a6a157700 1 -- 172.21.15.43:0/3403453661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a640728b0 con 0x7f2a641f2f80 2023-06-28T20:07:58.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.699+0000 7f2a5bfff700 1 -- 172.21.15.43:0/3403453661 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2a4c0075a0 con 0x7f2a641f2f80 2023-06-28T20:07:58.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.699+0000 7f2a5bfff700 1 --2- 172.21.15.43:0/3403453661 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2a4406ace0 0x7f2a4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:58.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.699+0000 7f2a5bfff700 1 -- 172.21.15.43:0/3403453661 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2a4c08a580 con 0x7f2a641f2f80 2023-06-28T20:07:58.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.699+0000 7f2a637fe700 1 --2- 172.21.15.43:0/3403453661 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2a4406ace0 0x7f2a4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:58.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.699+0000 7f2a637fe700 1 --2- 172.21.15.43:0/3403453661 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2a4406ace0 0x7f2a4406d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2a54009c20 tx=0x7f2a54008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:58.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.703+0000 7f2a5bfff700 1 -- 172.21.15.43:0/3403453661 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2a4c055640 con 0x7f2a641f2f80 2023-06-28T20:07:58.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.901+0000 7f2a6a157700 1 -- 172.21.15.43:0/3403453661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2a6406d9c0 con 0x7f2a641f2f80 2023-06-28T20:07:58.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.901+0000 7f2a5bfff700 1 -- 172.21.15.43:0/3403453661 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2a4c007d70 con 0x7f2a641f2f80 2023-06-28T20:07:58.904 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-06-28T20:07:58.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.903+0000 7f2a6a157700 1 -- 172.21.15.43:0/3403453661 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2a4406ace0 msgr2=0x7f2a4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:58.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.903+0000 7f2a6a157700 1 --2- 172.21.15.43:0/3403453661 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2a4406ace0 0x7f2a4406d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2a54009c20 tx=0x7f2a54008040 comp rx=0 tx=0).stop 2023-06-28T20:07:58.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.903+0000 7f2a6a157700 1 -- 172.21.15.43:0/3403453661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a641f2f80 msgr2=0x7f2a640ad140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:58.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.903+0000 7f2a6a157700 1 --2- 172.21.15.43:0/3403453661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a641f2f80 0x7f2a640ad140 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2a5000c210 tx=0x7f2a641603e0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.903+0000 7f2a6a157700 1 -- 172.21.15.43:0/3403453661 shutdown_connections 2023-06-28T20:07:58.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.903+0000 7f2a6a157700 1 --2- 172.21.15.43:0/3403453661 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2a4406ace0 0x7f2a4406d190 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.903+0000 7f2a6a157700 1 --2- 172.21.15.43:0/3403453661 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2a64150d10 0x7f2a64151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.903+0000 7f2a6a157700 1 --2- 172.21.15.43:0/3403453661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a50004850 0x7f2a640b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.903+0000 7f2a6a157700 1 --2- 172.21.15.43:0/3403453661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a641f2f80 0x7f2a640ad140 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:58.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.903+0000 7f2a6a157700 1 -- 172.21.15.43:0/3403453661 >> 172.21.15.43:0/3403453661 conn(0x7f2a6406b000 msgr2=0x7f2a640ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:58.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.904+0000 7f2a6a157700 1 -- 172.21.15.43:0/3403453661 shutdown_connections 2023-06-28T20:07:58.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:58.904+0000 7f2a6a157700 1 -- 172.21.15.43:0/3403453661 wait complete. 2023-06-28T20:07:58.921 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-28T20:07:59.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.044+0000 7faee0820700 1 Processor -- start 2023-06-28T20:07:59.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.044+0000 7faee0820700 1 -- start start 2023-06-28T20:07:59.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.045+0000 7faee0820700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faedc1596a0 0x7faedc15d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:59.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.045+0000 7faee0820700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faedc070b40 con 0x7faedc1592d0 2023-06-28T20:07:59.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.045+0000 7faee0820700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faedc070e20 con 0x7faedc0b20e0 2023-06-28T20:07:59.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.045+0000 7faee0820700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faedc071100 con 0x7faedc1596a0 2023-06-28T20:07:59.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.045+0000 7faeda59c700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7faedc1592d0 0x7faedc0b19d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43416/0 (socket says 172.21.15.43:43416) 2023-06-28T20:07:59.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.045+0000 7faeda59c700 1 -- 172.21.15.43:0/2900461856 learned_addr learned my addr 172.21.15.43:0/2900461856 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:59.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.045+0000 7faed959a700 1 -- 172.21.15.43:0/2900461856 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 982798476 0 0) 0x7faedc070b40 con 0x7faedc1592d0 2023-06-28T20:07:59.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.045+0000 7faed959a700 1 -- 172.21.15.43:0/2900461856 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faec40035b0 con 0x7faedc1592d0 2023-06-28T20:07:59.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.045+0000 7faedad9d700 1 --2- 172.21.15.43:0/2900461856 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faedc1596a0 0x7faedc15d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:59.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.045+0000 7faed959a700 1 -- 172.21.15.43:0/2900461856 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2242628609 0 0) 0x7faec40035b0 con 0x7faedc1592d0 2023-06-28T20:07:59.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.045+0000 7faed959a700 1 -- 172.21.15.43:0/2900461856 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faedc1596a0 msgr2=0x7faedc15d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:59.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.045+0000 7faed959a700 1 --2- 172.21.15.43:0/2900461856 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faedc1596a0 0x7faedc15d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:59.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.045+0000 7faed959a700 1 -- 172.21.15.43:0/2900461856 >> v1:172.21.15.178:6789/0 conn(0x7faedc0b20e0 legacy=0x7faedc15d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:59.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.045+0000 7faed959a700 1 -- 172.21.15.43:0/2900461856 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faedc0713e0 con 0x7faedc1592d0 2023-06-28T20:07:59.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.045+0000 7faed959a700 1 -- 172.21.15.43:0/2900461856 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7faecc001670 con 0x7faedc1592d0 2023-06-28T20:07:59.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.045+0000 7faedad9d700 1 --2- 172.21.15.43:0/2900461856 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faedc1596a0 0x7faedc15d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:59.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.045+0000 7faed959a700 1 -- 172.21.15.43:0/2900461856 >> v1:172.21.15.43:6789/0 conn(0x7faedc1592d0 legacy=0x7faedc0b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:59.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.046+0000 7faed959a700 1 --2- 172.21.15.43:0/2900461856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec4003a30 0x7faec4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:59.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.046+0000 7faed959a700 1 --2- 172.21.15.43:0/2900461856 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faedc1596a0 0x7faec4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:59.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.046+0000 7faed959a700 1 --2- 172.21.15.43:0/2900461856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faec4004d90 0x7faec4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:59.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.046+0000 7faed959a700 1 -- 172.21.15.43:0/2900461856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faedc071100 con 0x7faec4003a30 2023-06-28T20:07:59.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.046+0000 7faed959a700 1 -- 172.21.15.43:0/2900461856 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec400bb20 con 0x7faec4004d90 2023-06-28T20:07:59.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.046+0000 7faed959a700 1 -- 172.21.15.43:0/2900461856 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec400bce0 con 0x7faedc1596a0 2023-06-28T20:07:59.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.046+0000 7faeda59c700 1 --2- 172.21.15.43:0/2900461856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec4003a30 0x7faec4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:59.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.046+0000 7faed9d9b700 1 --2- 172.21.15.43:0/2900461856 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faedc1596a0 0x7faec4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:59.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.046+0000 7faedad9d700 1 --2- 172.21.15.43:0/2900461856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faec4004d90 0x7faec4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:59.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.046+0000 7faeda59c700 1 -- 172.21.15.43:0/2900461856 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faedc1596a0 msgr2=0x7faec4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:59.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.046+0000 7faeda59c700 1 --2- 172.21.15.43:0/2900461856 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faedc1596a0 0x7faec4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:59.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.046+0000 7faeda59c700 1 -- 172.21.15.43:0/2900461856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faec4004d90 msgr2=0x7faec4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:07:59.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.046+0000 7faeda59c700 1 --2- 172.21.15.43:0/2900461856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faec4004d90 0x7faec4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:59.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.046+0000 7faeda59c700 1 -- 172.21.15.43:0/2900461856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec40035b0 con 0x7faec4003a30 2023-06-28T20:07:59.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.046+0000 7faed9d9b700 1 --2- 172.21.15.43:0/2900461856 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faedc1596a0 0x7faec4004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:07:59.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.046+0000 7faeda59c700 1 --2- 172.21.15.43:0/2900461856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec4003a30 0x7faec4003e40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faecc00c240 tx=0x7faec4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3895ea19124b41f server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:59.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.046+0000 7faed959a700 1 -- 172.21.15.43:0/2900461856 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc007c10 con 0x7faec4003a30 2023-06-28T20:07:59.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.046+0000 7faed959a700 1 -- 172.21.15.43:0/2900461856 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faecc00ce00 con 0x7faec4003a30 2023-06-28T20:07:59.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.046+0000 7faed959a700 1 -- 172.21.15.43:0/2900461856 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc005ba0 con 0x7faec4003a30 2023-06-28T20:07:59.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.047+0000 7faee0820700 1 -- 172.21.15.43:0/2900461856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec4003a30 msgr2=0x7faec4003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:59.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.047+0000 7faee0820700 1 --2- 172.21.15.43:0/2900461856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faec4003a30 0x7faec4003e40 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faecc00c240 tx=0x7faec4005880 comp rx=0 tx=0).stop 2023-06-28T20:07:59.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.047+0000 7faedad9d700 1 -- 172.21.15.43:0/2900461856 reap_dead start 2023-06-28T20:07:59.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.047+0000 7faee0820700 1 -- 172.21.15.43:0/2900461856 shutdown_connections 2023-06-28T20:07:59.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.047+0000 7faee0820700 1 -- 172.21.15.43:0/2900461856 >> 172.21.15.43:0/2900461856 conn(0x7faedc06b000 msgr2=0x7faedc06b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:59.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.047+0000 7faee0820700 1 -- 172.21.15.43:0/2900461856 shutdown_connections 2023-06-28T20:07:59.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.047+0000 7faee0820700 1 -- 172.21.15.43:0/2900461856 wait complete. 2023-06-28T20:07:59.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.048+0000 7faee0820700 1 Processor -- start 2023-06-28T20:07:59.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.048+0000 7faee0820700 1 -- start start 2023-06-28T20:07:59.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.049+0000 7faee0820700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faec4003a30 0x7faedc14d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:59.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.049+0000 7faee0820700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faedc0b4de0 0x7faedc15a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:59.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.049+0000 7faee0820700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faedc1501e0 0x7faedc150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:59.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.049+0000 7faee0820700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faedc150bc0 con 0x7faedc0b4de0 2023-06-28T20:07:59.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.049+0000 7faee0820700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faedc15adb0 con 0x7faedc1501e0 2023-06-28T20:07:59.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.049+0000 7faee0820700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faedc15b090 con 0x7faec4003a30 2023-06-28T20:07:59.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.049+0000 7faed9d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faedc0b4de0 0x7faedc15a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:59.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.049+0000 7faed9d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faedc0b4de0 0x7faedc15a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54244/0 (socket says 172.21.15.43:54244) 2023-06-28T20:07:59.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.049+0000 7faed9d9b700 1 -- 172.21.15.43:0/1645373828 learned_addr learned my addr 172.21.15.43:0/1645373828 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:59.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.049+0000 7faeda59c700 1 --2- 172.21.15.43:0/1645373828 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faec4003a30 0x7faedc14d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:59.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.049+0000 7faedad9d700 1 --2- 172.21.15.43:0/1645373828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faedc1501e0 0x7faedc150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:59.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.049+0000 7faed9d9b700 1 -- 172.21.15.43:0/1645373828 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faec4003a30 msgr2=0x7faedc14d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:59.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.049+0000 7faed9d9b700 1 --2- 172.21.15.43:0/1645373828 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faec4003a30 0x7faedc14d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:59.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.049+0000 7faed9d9b700 1 -- 172.21.15.43:0/1645373828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faedc1501e0 msgr2=0x7faedc150680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:59.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.049+0000 7faed9d9b700 1 --2- 172.21.15.43:0/1645373828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faedc1501e0 0x7faedc150680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:59.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.049+0000 7faed9d9b700 1 -- 172.21.15.43:0/1645373828 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faedc15b370 con 0x7faedc0b4de0 2023-06-28T20:07:59.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.050+0000 7faed9d9b700 1 --2- 172.21.15.43:0/1645373828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faedc0b4de0 0x7faedc15a4b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7faed000e780 tx=0x7faed000eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:59.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.050+0000 7faecaffd700 1 -- 172.21.15.43:0/1645373828 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed00052e0 con 0x7faedc0b4de0 2023-06-28T20:07:59.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.050+0000 7faecaffd700 1 -- 172.21.15.43:0/1645373828 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed0005440 con 0x7faedc0b4de0 2023-06-28T20:07:59.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.050+0000 7faee0820700 1 -- 172.21.15.43:0/1645373828 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faedc14e250 con 0x7faedc0b4de0 2023-06-28T20:07:59.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.050+0000 7faecaffd700 1 -- 172.21.15.43:0/1645373828 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed00038a0 con 0x7faedc0b4de0 2023-06-28T20:07:59.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.050+0000 7faee0820700 1 -- 172.21.15.43:0/1645373828 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faedc14d6c0 con 0x7faedc0b4de0 2023-06-28T20:07:59.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.051+0000 7faee0820700 1 -- 172.21.15.43:0/1645373828 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faedc14e8a0 con 0x7faedc0b4de0 2023-06-28T20:07:59.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.051+0000 7faecaffd700 1 -- 172.21.15.43:0/1645373828 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faed0003a00 con 0x7faedc0b4de0 2023-06-28T20:07:59.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.052+0000 7faecaffd700 1 --2- 172.21.15.43:0/1645373828 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7faebc06ace0 0x7faebc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:59.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.052+0000 7faecaffd700 1 -- 172.21.15.43:0/1645373828 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faed0088ba0 con 0x7faedc0b4de0 2023-06-28T20:07:59.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.052+0000 7faeda59c700 1 --2- 172.21.15.43:0/1645373828 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7faebc06ace0 0x7faebc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:59.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.052+0000 7faeda59c700 1 --2- 172.21.15.43:0/1645373828 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7faebc06ace0 0x7faebc06d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faedc150190 tx=0x7faecc007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:59.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.056+0000 7faecaffd700 1 -- 172.21.15.43:0/1645373828 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7faed0053d10 con 0x7faedc0b4de0 2023-06-28T20:07:59.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.271+0000 7faee0820700 1 -- 172.21.15.43:0/1645373828 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faedc0cc3e0 con 0x7faedc0b4de0 2023-06-28T20:07:59.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.272+0000 7faecaffd700 1 -- 172.21.15.43:0/1645373828 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7faed0003cb0 con 0x7faedc0b4de0 2023-06-28T20:07:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.273+0000 7faee0820700 1 -- 172.21.15.43:0/1645373828 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7faebc06ace0 msgr2=0x7faebc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.273+0000 7faee0820700 1 --2- 172.21.15.43:0/1645373828 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7faebc06ace0 0x7faebc06d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faedc150190 tx=0x7faecc007000 comp rx=0 tx=0).stop 2023-06-28T20:07:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.273+0000 7faee0820700 1 -- 172.21.15.43:0/1645373828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faedc0b4de0 msgr2=0x7faedc15a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.273+0000 7faee0820700 1 --2- 172.21.15.43:0/1645373828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faedc0b4de0 0x7faedc15a4b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7faed000e780 tx=0x7faed000eb40 comp rx=0 tx=0).stop 2023-06-28T20:07:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.273+0000 7faee0820700 1 -- 172.21.15.43:0/1645373828 shutdown_connections 2023-06-28T20:07:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.273+0000 7faee0820700 1 --2- 172.21.15.43:0/1645373828 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faedc1501e0 0x7faedc150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.273+0000 7faee0820700 1 --2- 172.21.15.43:0/1645373828 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7faebc06ace0 0x7faebc06d190 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.273+0000 7faee0820700 1 --2- 172.21.15.43:0/1645373828 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7faec4003a30 0x7faedc14d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.273+0000 7faee0820700 1 --2- 172.21.15.43:0/1645373828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faedc0b4de0 0x7faedc15a4b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.274+0000 7faee0820700 1 -- 172.21.15.43:0/1645373828 >> 172.21.15.43:0/1645373828 conn(0x7faedc06b000 msgr2=0x7faedc15cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.274+0000 7faee0820700 1 -- 172.21.15.43:0/1645373828 shutdown_connections 2023-06-28T20:07:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.274+0000 7faee0820700 1 -- 172.21.15.43:0/1645373828 wait complete. 2023-06-28T20:07:59.279 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2023-06-28T20:07:59.289 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-06-28T20:07:59.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:59.290 DEBUG:teuthology.orchestra.run.smithi178:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-06-28T20:07:59.335 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi178.front.sepia.ceph.com 2023-06-28T20:07:59.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:07:59.336 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-06-28T20:07:59.412 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:07:59.413 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:07:59.433 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:07:59.434 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:07:59.490 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:07:59.490 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:07:59.549 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:07:59.718 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.711+0000 7f23a219b700 1 Processor -- start 2023-06-28T20:07:59.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.711+0000 7f23a219b700 1 -- start start 2023-06-28T20:07:59.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.712+0000 7f23a219b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f239c06c8b0 0x7f239c15e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:59.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.712+0000 7f23a219b700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f239c15def0 con 0x7f239c15d680 2023-06-28T20:07:59.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.712+0000 7f23a219b700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f239c0cf710 con 0x7f239c15c4e0 2023-06-28T20:07:59.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.712+0000 7f23a219b700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f239c0cf150 con 0x7f239c06c8b0 2023-06-28T20:07:59.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.712+0000 7f239affd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f239c06c8b0 0x7f239c15e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:59.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.712+0000 7f239bfff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f239c15d680 0x7f239c15e8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43440/0 (socket says 172.21.15.43:43440) 2023-06-28T20:07:59.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.712+0000 7f239bfff700 1 -- 172.21.15.43:0/2472622525 learned_addr learned my addr 172.21.15.43:0/2472622525 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:07:59.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.713+0000 7f239affd700 1 -- 172.21.15.43:0/2472622525 >> v1:172.21.15.178:6789/0 conn(0x7f239c15c4e0 legacy=0x7f239c06a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:59.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.713+0000 7f239affd700 1 -- 172.21.15.43:0/2472622525 >> v1:172.21.15.43:6789/0 conn(0x7f239c15d680 legacy=0x7f239c15e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:59.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.713+0000 7f239affd700 1 -- 172.21.15.43:0/2472622525 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f239c0cf430 con 0x7f239c06c8b0 2023-06-28T20:07:59.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.713+0000 7f239a7fc700 1 -- 172.21.15.43:0/2472622525 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1016930085 0 0) 0x7f239c15def0 con 0x7f239c15d680 2023-06-28T20:07:59.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.713+0000 7f239affd700 1 --2- 172.21.15.43:0/2472622525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f239c06c8b0 0x7f239c15e380 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f238c009840 tx=0x7f238c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d93cf3f0fe3aa2 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:59.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.713+0000 7f239a7fc700 1 -- 172.21.15.43:0/2472622525 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c011830 con 0x7f239c06c8b0 2023-06-28T20:07:59.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.713+0000 7f239a7fc700 1 -- 172.21.15.43:0/2472622525 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238c011990 con 0x7f239c06c8b0 2023-06-28T20:07:59.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.713+0000 7f239a7fc700 1 -- 172.21.15.43:0/2472622525 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c011d20 con 0x7f239c06c8b0 2023-06-28T20:07:59.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.714+0000 7f23a219b700 1 -- 172.21.15.43:0/2472622525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f239c06c8b0 msgr2=0x7f239c15e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:59.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.714+0000 7f23a219b700 1 --2- 172.21.15.43:0/2472622525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f239c06c8b0 0x7f239c15e380 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f238c009840 tx=0x7f238c009b50 comp rx=0 tx=0).stop 2023-06-28T20:07:59.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.714+0000 7f23a219b700 1 -- 172.21.15.43:0/2472622525 shutdown_connections 2023-06-28T20:07:59.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.714+0000 7f23a219b700 1 --2- 172.21.15.43:0/2472622525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f239c06c8b0 0x7f239c15e380 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:59.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.714+0000 7f23a219b700 1 -- 172.21.15.43:0/2472622525 >> 172.21.15.43:0/2472622525 conn(0x7f239c0d2eb0 msgr2=0x7f239c0d32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:59.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.714+0000 7f23a219b700 1 -- 172.21.15.43:0/2472622525 shutdown_connections 2023-06-28T20:07:59.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.714+0000 7f23a219b700 1 -- 172.21.15.43:0/2472622525 wait complete. 2023-06-28T20:07:59.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.715+0000 7f23a219b700 1 Processor -- start 2023-06-28T20:07:59.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.715+0000 7f23a219b700 1 -- start start 2023-06-28T20:07:59.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.716+0000 7f23a219b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f239c06c8b0 0x7f239c0ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:59.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.716+0000 7f23a219b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f239c15c4e0 0x7f239c0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:59.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.716+0000 7f23a219b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f239c15d680 0x7f239c13f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:59.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.716+0000 7f23a219b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c0b3b10 con 0x7f239c15c4e0 2023-06-28T20:07:59.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.716+0000 7f23a219b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c0b3df0 con 0x7f239c15d680 2023-06-28T20:07:59.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.716+0000 7f23a219b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f239c0b40d0 con 0x7f239c06c8b0 2023-06-28T20:07:59.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.716+0000 7f239affd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f239c15c4e0 0x7f239c0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:59.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.716+0000 7f239affd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f239c15c4e0 0x7f239c0b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54256/0 (socket says 172.21.15.43:54256) 2023-06-28T20:07:59.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.716+0000 7f239affd700 1 -- 172.21.15.43:0/339540695 learned_addr learned my addr 172.21.15.43:0/339540695 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:07:59.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.716+0000 7f239bfff700 1 --2- 172.21.15.43:0/339540695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f239c15d680 0x7f239c13f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:59.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.716+0000 7f239b7fe700 1 --2- 172.21.15.43:0/339540695 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f239c06c8b0 0x7f239c0ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:59.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.716+0000 7f239affd700 1 -- 172.21.15.43:0/339540695 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f239c06c8b0 msgr2=0x7f239c0ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:59.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.716+0000 7f239affd700 1 --2- 172.21.15.43:0/339540695 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f239c06c8b0 0x7f239c0ccaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:59.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.716+0000 7f239affd700 1 -- 172.21.15.43:0/339540695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f239c15d680 msgr2=0x7f239c13f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:59.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.716+0000 7f239affd700 1 --2- 172.21.15.43:0/339540695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f239c15d680 0x7f239c13f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:59.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.716+0000 7f239affd700 1 -- 172.21.15.43:0/339540695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f238c0094a0 con 0x7f239c15c4e0 2023-06-28T20:07:59.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.716+0000 7f239b7fe700 1 --2- 172.21.15.43:0/339540695 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f239c06c8b0 0x7f239c0ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:59.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.716+0000 7f239bfff700 1 --2- 172.21.15.43:0/339540695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f239c15d680 0x7f239c13f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:07:59.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.716+0000 7f239affd700 1 --2- 172.21.15.43:0/339540695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f239c15c4e0 0x7f239c0b4bb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f238c014040 tx=0x7f238c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:59.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.716+0000 7f2393fff700 1 -- 172.21.15.43:0/339540695 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c01a980 con 0x7f239c15c4e0 2023-06-28T20:07:59.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.717+0000 7f2393fff700 1 -- 172.21.15.43:0/339540695 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238c01ab00 con 0x7f239c15c4e0 2023-06-28T20:07:59.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.717+0000 7f23a219b700 1 -- 172.21.15.43:0/339540695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f239c0b43b0 con 0x7f239c15c4e0 2023-06-28T20:07:59.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.717+0000 7f2393fff700 1 -- 172.21.15.43:0/339540695 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c024420 con 0x7f239c15c4e0 2023-06-28T20:07:59.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.717+0000 7f23a219b700 1 -- 172.21.15.43:0/339540695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f239c0ccfe0 con 0x7f239c15c4e0 2023-06-28T20:07:59.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.718+0000 7f23a219b700 1 -- 172.21.15.43:0/339540695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f239c0ce2e0 con 0x7f239c15c4e0 2023-06-28T20:07:59.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.718+0000 7f2393fff700 1 -- 172.21.15.43:0/339540695 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f238c022070 con 0x7f239c15c4e0 2023-06-28T20:07:59.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.718+0000 7f2393fff700 1 --2- 172.21.15.43:0/339540695 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f237c06ace0 0x7f237c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:07:59.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.718+0000 7f2393fff700 1 -- 172.21.15.43:0/339540695 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f238c08a5f0 con 0x7f239c15c4e0 2023-06-28T20:07:59.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.719+0000 7f239b7fe700 1 --2- 172.21.15.43:0/339540695 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f237c06ace0 0x7f237c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:07:59.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.719+0000 7f239b7fe700 1 --2- 172.21.15.43:0/339540695 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f237c06ace0 0x7f237c06d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2384000f60 tx=0x7f2384009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:07:59.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.723+0000 7f2393fff700 1 -- 172.21.15.43:0/339540695 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f238c055730 con 0x7f239c15c4e0 2023-06-28T20:07:59.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.919+0000 7f23a219b700 1 -- 172.21.15.43:0/339540695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f239c0728b0 con 0x7f239c15c4e0 2023-06-28T20:07:59.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.920+0000 7f2393fff700 1 -- 172.21.15.43:0/339540695 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18221 (secure 0 0 0) 0x7f238c058d50 con 0x7f239c15c4e0 2023-06-28T20:07:59.923 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:07:59.924 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:07:22.377248+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:07:18.624759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-28T20:07:19.760017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:07:59.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.922+0000 7f23a219b700 1 -- 172.21.15.43:0/339540695 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f237c06ace0 msgr2=0x7f237c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:59.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.922+0000 7f23a219b700 1 --2- 172.21.15.43:0/339540695 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f237c06ace0 0x7f237c06d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2384000f60 tx=0x7f2384009040 comp rx=0 tx=0).stop 2023-06-28T20:07:59.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.922+0000 7f23a219b700 1 -- 172.21.15.43:0/339540695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f239c15c4e0 msgr2=0x7f239c0b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:07:59.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.922+0000 7f23a219b700 1 --2- 172.21.15.43:0/339540695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f239c15c4e0 0x7f239c0b4bb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f238c014040 tx=0x7f238c004750 comp rx=0 tx=0).stop 2023-06-28T20:07:59.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.922+0000 7f23a219b700 1 -- 172.21.15.43:0/339540695 shutdown_connections 2023-06-28T20:07:59.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.922+0000 7f23a219b700 1 --2- 172.21.15.43:0/339540695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f239c15d680 0x7f239c13f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:59.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.922+0000 7f23a219b700 1 --2- 172.21.15.43:0/339540695 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f237c06ace0 0x7f237c06d190 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:59.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.922+0000 7f23a219b700 1 --2- 172.21.15.43:0/339540695 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f239c06c8b0 0x7f239c0ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:59.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.922+0000 7f23a219b700 1 --2- 172.21.15.43:0/339540695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f239c15c4e0 0x7f239c0b4bb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:07:59.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.922+0000 7f23a219b700 1 -- 172.21.15.43:0/339540695 >> 172.21.15.43:0/339540695 conn(0x7f239c0d2eb0 msgr2=0x7f239c15b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:07:59.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.922+0000 7f23a219b700 1 -- 172.21.15.43:0/339540695 shutdown_connections 2023-06-28T20:07:59.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:07:59.922+0000 7f23a219b700 1 -- 172.21.15.43:0/339540695 wait complete. 2023-06-28T20:07:59.938 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-28T20:07:13.580969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-28T20:07:14.038358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-28T20:07:18.624759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-28T20:07:19.760017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-28T20:07:59.940 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-28T20:08:00.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.065+0000 7f809c454700 1 Processor -- start 2023-06-28T20:08:00.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.065+0000 7f809c454700 1 -- start start 2023-06-28T20:08:00.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.066+0000 7f809c454700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f809406a830 0x7f80940c9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:00.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.066+0000 7f809c454700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80940ce370 con 0x7f809406b420 2023-06-28T20:08:00.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.066+0000 7f809c454700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f809406b080 con 0x7f809406ac00 2023-06-28T20:08:00.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.066+0000 7f809c454700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f809406b8a0 con 0x7f809406a830 2023-06-28T20:08:00.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.066+0000 7f809a9f1700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f809406b420 0x7f809406c0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43454/0 (socket says 172.21.15.43:43454) 2023-06-28T20:08:00.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.066+0000 7f809a9f1700 1 -- 172.21.15.43:0/575007507 learned_addr learned my addr 172.21.15.43:0/575007507 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.066+0000 7f809a1f0700 1 --2- 172.21.15.43:0/575007507 >> v2:172.21.15.178:3300/0 conn(0x7f809406a830 0x7f80940c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.066+0000 7f80991ee700 1 -- 172.21.15.43:0/575007507 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4084106225 0 0) 0x7f80940ce370 con 0x7f809406b420 2023-06-28T20:08:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.066+0000 7f80991ee700 1 -- 172.21.15.43:0/575007507 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80800035b0 con 0x7f809406b420 2023-06-28T20:08:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.066+0000 7f80991ee700 1 -- 172.21.15.43:0/575007507 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4289577203 0 0) 0x7f80800035b0 con 0x7f809406b420 2023-06-28T20:08:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.066+0000 7f80991ee700 1 -- 172.21.15.43:0/575007507 >> v2:172.21.15.178:3300/0 conn(0x7f809406a830 msgr2=0x7f80940c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.066+0000 7f80991ee700 1 --2- 172.21.15.43:0/575007507 >> v2:172.21.15.178:3300/0 conn(0x7f809406a830 0x7f80940c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.066+0000 7f80991ee700 1 -- 172.21.15.43:0/575007507 >> v1:172.21.15.178:6789/0 conn(0x7f809406ac00 legacy=0x7f8094069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.066+0000 7f80991ee700 1 -- 172.21.15.43:0/575007507 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80940ce610 con 0x7f809406b420 2023-06-28T20:08:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.066+0000 7f80991ee700 1 -- 172.21.15.43:0/575007507 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f808c0019f0 con 0x7f809406b420 2023-06-28T20:08:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.067+0000 7f80991ee700 1 -- 172.21.15.43:0/575007507 >> v1:172.21.15.43:6789/0 conn(0x7f809406b420 legacy=0x7f809406c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.067+0000 7f80991ee700 1 --2- 172.21.15.43:0/575007507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8080003a00 0x7f8080003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.067+0000 7f80991ee700 1 --2- 172.21.15.43:0/575007507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8080004850 0x7f8080004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.067+0000 7f80991ee700 1 --2- 172.21.15.43:0/575007507 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8080005200 0x7f808000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.067+0000 7f80991ee700 1 -- 172.21.15.43:0/575007507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809406b8a0 con 0x7f8080004850 2023-06-28T20:08:00.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.067+0000 7f80991ee700 1 -- 172.21.15.43:0/575007507 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808000c110 con 0x7f8080003a00 2023-06-28T20:08:00.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.067+0000 7f80991ee700 1 -- 172.21.15.43:0/575007507 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808000c2d0 con 0x7f8080005200 2023-06-28T20:08:00.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.067+0000 7f80999ef700 1 --2- 172.21.15.43:0/575007507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8080004850 0x7f8080004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:00.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.067+0000 7f809a1f0700 1 --2- 172.21.15.43:0/575007507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8080003a00 0x7f8080003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:00.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.067+0000 7f809a9f1700 1 --2- 172.21.15.43:0/575007507 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8080005200 0x7f808000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:00.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.067+0000 7f80999ef700 1 -- 172.21.15.43:0/575007507 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8080005200 msgr2=0x7f808000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:00.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.067+0000 7f80999ef700 1 --2- 172.21.15.43:0/575007507 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8080005200 0x7f808000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.067+0000 7f80999ef700 1 -- 172.21.15.43:0/575007507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8080003a00 msgr2=0x7f8080003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:00.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.067+0000 7f80999ef700 1 --2- 172.21.15.43:0/575007507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8080003a00 0x7f8080003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.067+0000 7f80999ef700 1 -- 172.21.15.43:0/575007507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80800035b0 con 0x7f8080004850 2023-06-28T20:08:00.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.067+0000 7f809a9f1700 1 -- 172.21.15.43:0/575007507 reap_dead start 2023-06-28T20:08:00.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.067+0000 7f80999ef700 1 --2- 172.21.15.43:0/575007507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8080004850 0x7f8080004cc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f80840017c0 tx=0x7f8080005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e834cad0230ed072 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:00.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.067+0000 7f80991ee700 1 -- 172.21.15.43:0/575007507 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80840052d0 con 0x7f8080004850 2023-06-28T20:08:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.067+0000 7f80991ee700 1 -- 172.21.15.43:0/575007507 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8084005430 con 0x7f8080004850 2023-06-28T20:08:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.067+0000 7f80991ee700 1 -- 172.21.15.43:0/575007507 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80840038c0 con 0x7f8080004850 2023-06-28T20:08:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.068+0000 7f809c454700 1 -- 172.21.15.43:0/575007507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8080004850 msgr2=0x7f8080004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.068+0000 7f809c454700 1 --2- 172.21.15.43:0/575007507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8080004850 0x7f8080004cc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f80840017c0 tx=0x7f8080005880 comp rx=0 tx=0).stop 2023-06-28T20:08:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.068+0000 7f809c454700 1 -- 172.21.15.43:0/575007507 shutdown_connections 2023-06-28T20:08:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.068+0000 7f809c454700 1 --2- 172.21.15.43:0/575007507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8080004850 0x7f8080004cc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.068+0000 7f809c454700 1 -- 172.21.15.43:0/575007507 >> 172.21.15.43:0/575007507 conn(0x7f80940b1a50 msgr2=0x7f80940cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.068+0000 7f809c454700 1 -- 172.21.15.43:0/575007507 shutdown_connections 2023-06-28T20:08:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.068+0000 7f809c454700 1 -- 172.21.15.43:0/575007507 wait complete. 2023-06-28T20:08:00.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.069+0000 7f809c454700 1 Processor -- start 2023-06-28T20:08:00.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.069+0000 7f809c454700 1 -- start start 2023-06-28T20:08:00.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.070+0000 7f809c454700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8080004850 0x7f80940b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:00.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.070+0000 7f809c454700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80941eea70 0x7f809406c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:00.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.070+0000 7f809c454700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f809414c4b0 0x7f809414c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:00.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.070+0000 7f809c454700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80941f1f90 con 0x7f809414c4b0 2023-06-28T20:08:00.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.070+0000 7f809c454700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809414ce40 con 0x7f80941eea70 2023-06-28T20:08:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.070+0000 7f809c454700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80940b5000 con 0x7f8080004850 2023-06-28T20:08:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.070+0000 7f809a9f1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f809414c4b0 0x7f809414c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.070+0000 7f809a9f1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f809414c4b0 0x7f809414c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54280/0 (socket says 172.21.15.43:54280) 2023-06-28T20:08:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.070+0000 7f809a9f1700 1 -- 172.21.15.43:0/3797671587 learned_addr learned my addr 172.21.15.43:0/3797671587 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.070+0000 7f809a1f0700 1 --2- 172.21.15.43:0/3797671587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8080004850 0x7f80940b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.070+0000 7f80999ef700 1 --2- 172.21.15.43:0/3797671587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80941eea70 0x7f809406c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.070+0000 7f809a9f1700 1 -- 172.21.15.43:0/3797671587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8080004850 msgr2=0x7f80940b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.070+0000 7f809a9f1700 1 --2- 172.21.15.43:0/3797671587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8080004850 0x7f80940b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.070+0000 7f809a9f1700 1 -- 172.21.15.43:0/3797671587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80941eea70 msgr2=0x7f809406c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.070+0000 7f809a9f1700 1 --2- 172.21.15.43:0/3797671587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80941eea70 0x7f809406c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.070+0000 7f809a9f1700 1 -- 172.21.15.43:0/3797671587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80941f1660 con 0x7f809414c4b0 2023-06-28T20:08:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.070+0000 7f809a1f0700 1 --2- 172.21.15.43:0/3797671587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8080004850 0x7f80940b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.070+0000 7f80999ef700 1 --2- 172.21.15.43:0/3797671587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80941eea70 0x7f809406c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:00.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.071+0000 7f809a9f1700 1 --2- 172.21.15.43:0/3797671587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f809414c4b0 0x7f809414c900 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f80940caa10 tx=0x7f808c00a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.071+0000 7f808affd700 1 -- 172.21.15.43:0/3797671587 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c00adc0 con 0x7f809414c4b0 2023-06-28T20:08:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.071+0000 7f808affd700 1 -- 172.21.15.43:0/3797671587 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808c0133f0 con 0x7f809414c4b0 2023-06-28T20:08:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.071+0000 7f809c454700 1 -- 172.21.15.43:0/3797671587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80941f1940 con 0x7f809414c4b0 2023-06-28T20:08:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.071+0000 7f808affd700 1 -- 172.21.15.43:0/3797671587 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808c0137a0 con 0x7f809414c4b0 2023-06-28T20:08:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.071+0000 7f809c454700 1 -- 172.21.15.43:0/3797671587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80941eeee0 con 0x7f809414c4b0 2023-06-28T20:08:00.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.072+0000 7f809c454700 1 -- 172.21.15.43:0/3797671587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80940728b0 con 0x7f809414c4b0 2023-06-28T20:08:00.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.072+0000 7f808affd700 1 -- 172.21.15.43:0/3797671587 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f808c013900 con 0x7f809414c4b0 2023-06-28T20:08:00.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.073+0000 7f808affd700 1 --2- 172.21.15.43:0/3797671587 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f807806ace0 0x7f807806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:00.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.073+0000 7f808affd700 1 -- 172.21.15.43:0/3797671587 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f808c089db0 con 0x7f809414c4b0 2023-06-28T20:08:00.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.073+0000 7f809a1f0700 1 --2- 172.21.15.43:0/3797671587 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f807806ace0 0x7f807806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:00.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.073+0000 7f809a1f0700 1 --2- 172.21.15.43:0/3797671587 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f807806ace0 0x7f807806d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f809000abc0 tx=0x7f8090008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:00.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.077+0000 7f808affd700 1 -- 172.21.15.43:0/3797671587 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f808c054e60 con 0x7f809414c4b0 2023-06-28T20:08:00.278 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.275+0000 7f809c454700 1 -- 172.21.15.43:0/3797671587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f80941f20d0 con 0x7f809414c4b0 2023-06-28T20:08:00.279 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.276+0000 7f808affd700 1 -- 172.21.15.43:0/3797671587 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f808c058480 con 0x7f809414c4b0 2023-06-28T20:08:00.279 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-06-28T20:08:00.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.278+0000 7f809c454700 1 -- 172.21.15.43:0/3797671587 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f807806ace0 msgr2=0x7f807806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:00.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.278+0000 7f809c454700 1 --2- 172.21.15.43:0/3797671587 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f807806ace0 0x7f807806d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f809000abc0 tx=0x7f8090008040 comp rx=0 tx=0).stop 2023-06-28T20:08:00.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.278+0000 7f809c454700 1 -- 172.21.15.43:0/3797671587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f809414c4b0 msgr2=0x7f809414c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:00.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.278+0000 7f809c454700 1 --2- 172.21.15.43:0/3797671587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f809414c4b0 0x7f809414c900 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f80940caa10 tx=0x7f808c00a5f0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.281 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.278+0000 7f809c454700 1 -- 172.21.15.43:0/3797671587 shutdown_connections 2023-06-28T20:08:00.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.278+0000 7f809c454700 1 --2- 172.21.15.43:0/3797671587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f809414c4b0 0x7f809414c900 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.278+0000 7f809c454700 1 --2- 172.21.15.43:0/3797671587 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f807806ace0 0x7f807806d190 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.278+0000 7f809c454700 1 --2- 172.21.15.43:0/3797671587 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8080004850 0x7f80940b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.278+0000 7f809c454700 1 --2- 172.21.15.43:0/3797671587 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f80941eea70 0x7f809406c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.278+0000 7f809c454700 1 -- 172.21.15.43:0/3797671587 >> 172.21.15.43:0/3797671587 conn(0x7f80940b1a50 msgr2=0x7f8094156f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:00.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.278+0000 7f809c454700 1 -- 172.21.15.43:0/3797671587 shutdown_connections 2023-06-28T20:08:00.282 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.278+0000 7f809c454700 1 -- 172.21.15.43:0/3797671587 wait complete. 2023-06-28T20:08:00.293 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-28T20:08:00.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.418+0000 7fe3fc156700 1 Processor -- start 2023-06-28T20:08:00.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.418+0000 7fe3fc156700 1 -- start start 2023-06-28T20:08:00.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.419+0000 7fe3fc156700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3f40af060 0x7fe3f4160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:00.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.419+0000 7fe3fc156700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe3f406ccc0 0x7fe3f40b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:00.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.419+0000 7fe3fc156700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f4160250 con 0x7fe3f40aec90 2023-06-28T20:08:00.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.419+0000 7fe3fc156700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f415cb30 con 0x7fe3f406ccc0 2023-06-28T20:08:00.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.419+0000 7fe3fc156700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f4071410 con 0x7fe3f40af060 2023-06-28T20:08:00.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.419+0000 7fe3f9ef2700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe3f40aec90 0x7fe3f4070bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43470/0 (socket says 172.21.15.43:43470) 2023-06-28T20:08:00.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.419+0000 7fe3f9ef2700 1 -- 172.21.15.43:0/3365490241 learned_addr learned my addr 172.21.15.43:0/3365490241 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:00.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f8ef0700 1 -- 172.21.15.43:0/3365490241 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2496209453 0 0) 0x7fe3f4160250 con 0x7fe3f40aec90 2023-06-28T20:08:00.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f96f1700 1 --2- 172.21.15.43:0/3365490241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3f40af060 0x7fe3f4160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:00.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3fa6f3700 1 --2- 172.21.15.43:0/3365490241 >> v2:172.21.15.178:3300/0 conn(0x7fe3f406ccc0 0x7fe3f40b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:00.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f8ef0700 1 -- 172.21.15.43:0/3365490241 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3e00035b0 con 0x7fe3f40aec90 2023-06-28T20:08:00.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f8ef0700 1 -- 172.21.15.43:0/3365490241 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1868469229 0 0) 0x7fe3e00035b0 con 0x7fe3f40aec90 2023-06-28T20:08:00.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f8ef0700 1 -- 172.21.15.43:0/3365490241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3f40af060 msgr2=0x7fe3f4160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:00.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f8ef0700 1 --2- 172.21.15.43:0/3365490241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3f40af060 0x7fe3f4160cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f8ef0700 1 -- 172.21.15.43:0/3365490241 >> v2:172.21.15.178:3300/0 conn(0x7fe3f406ccc0 msgr2=0x7fe3f40b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:00.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f8ef0700 1 --2- 172.21.15.43:0/3365490241 >> v2:172.21.15.178:3300/0 conn(0x7fe3f406ccc0 0x7fe3f40b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f8ef0700 1 -- 172.21.15.43:0/3365490241 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f41604f0 con 0x7fe3f40aec90 2023-06-28T20:08:00.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f8ef0700 1 -- 172.21.15.43:0/3365490241 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7fe3e4001670 con 0x7fe3f40aec90 2023-06-28T20:08:00.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3fa6f3700 1 --2- 172.21.15.43:0/3365490241 >> v2:172.21.15.178:3300/0 conn(0x7fe3f406ccc0 0x7fe3f40b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:00.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f8ef0700 1 -- 172.21.15.43:0/3365490241 >> v1:172.21.15.43:6789/0 conn(0x7fe3f40aec90 legacy=0x7fe3f4070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:00.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f8ef0700 1 --2- 172.21.15.43:0/3365490241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3e0003a00 0x7fe3e0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:00.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f8ef0700 1 --2- 172.21.15.43:0/3365490241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3e0004430 0x7fe3e00048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:00.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f8ef0700 1 --2- 172.21.15.43:0/3365490241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3f40af060 0x7fe3e0005070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:00.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f8ef0700 1 -- 172.21.15.43:0/3365490241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f415cb30 con 0x7fe3e0003a00 2023-06-28T20:08:00.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f8ef0700 1 -- 172.21.15.43:0/3365490241 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f4071410 con 0x7fe3e0004430 2023-06-28T20:08:00.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f8ef0700 1 -- 172.21.15.43:0/3365490241 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e000ba10 con 0x7fe3f40af060 2023-06-28T20:08:00.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f9ef2700 1 --2- 172.21.15.43:0/3365490241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3e0003a00 0x7fe3e0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:00.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f96f1700 1 --2- 172.21.15.43:0/3365490241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3e0004430 0x7fe3e00048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:00.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3fa6f3700 1 --2- 172.21.15.43:0/3365490241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3f40af060 0x7fe3e0005070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:00.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f9ef2700 1 -- 172.21.15.43:0/3365490241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3f40af060 msgr2=0x7fe3e0005070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:00.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f9ef2700 1 --2- 172.21.15.43:0/3365490241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3f40af060 0x7fe3e0005070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f9ef2700 1 -- 172.21.15.43:0/3365490241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3e0004430 msgr2=0x7fe3e00048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:00.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f9ef2700 1 --2- 172.21.15.43:0/3365490241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3e0004430 0x7fe3e00048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.420+0000 7fe3f9ef2700 1 -- 172.21.15.43:0/3365490241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e00035b0 con 0x7fe3e0003a00 2023-06-28T20:08:00.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.421+0000 7fe3f9ef2700 1 --2- 172.21.15.43:0/3365490241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3e0003a00 0x7fe3e0003e10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400c320 tx=0x7fe3e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1abbfa4bef4d4f server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:00.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.421+0000 7fe3f8ef0700 1 -- 172.21.15.43:0/3365490241 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e400ce60 con 0x7fe3e0003a00 2023-06-28T20:08:00.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.421+0000 7fe3f8ef0700 1 -- 172.21.15.43:0/3365490241 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e4005960 con 0x7fe3e0003a00 2023-06-28T20:08:00.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.421+0000 7fe3f8ef0700 1 -- 172.21.15.43:0/3365490241 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4005d10 con 0x7fe3e0003a00 2023-06-28T20:08:00.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.421+0000 7fe3fc156700 1 -- 172.21.15.43:0/3365490241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3e0003a00 msgr2=0x7fe3e0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:00.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.421+0000 7fe3fc156700 1 --2- 172.21.15.43:0/3365490241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3e0003a00 0x7fe3e0003e10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400c320 tx=0x7fe3e0005880 comp rx=0 tx=0).stop 2023-06-28T20:08:00.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.421+0000 7fe3fa6f3700 1 -- 172.21.15.43:0/3365490241 reap_dead start 2023-06-28T20:08:00.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.421+0000 7fe3fc156700 1 -- 172.21.15.43:0/3365490241 shutdown_connections 2023-06-28T20:08:00.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.421+0000 7fe3fc156700 1 -- 172.21.15.43:0/3365490241 >> 172.21.15.43:0/3365490241 conn(0x7fe3f406b000 msgr2=0x7fe3f406b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:00.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.421+0000 7fe3fc156700 1 -- 172.21.15.43:0/3365490241 shutdown_connections 2023-06-28T20:08:00.430 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.421+0000 7fe3fc156700 1 -- 172.21.15.43:0/3365490241 wait complete. 2023-06-28T20:08:00.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.423+0000 7fe3fc156700 1 Processor -- start 2023-06-28T20:08:00.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.423+0000 7fe3fc156700 1 -- start start 2023-06-28T20:08:00.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.424+0000 7fe3fc156700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3e0003a00 0x7fe3f40b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:00.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.424+0000 7fe3fc156700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3f40b3e60 0x7fe3f4146dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:00.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.424+0000 7fe3fc156700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3f40b0600 0x7fe3f40b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:00.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.424+0000 7fe3fc156700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f40b42d0 con 0x7fe3f40b3e60 2023-06-28T20:08:00.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.424+0000 7fe3fc156700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f40b0fb0 con 0x7fe3e0003a00 2023-06-28T20:08:00.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.424+0000 7fe3fc156700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f415cc70 con 0x7fe3f40b0600 2023-06-28T20:08:00.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.424+0000 7fe3f96f1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3f40b3e60 0x7fe3f4146dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:00.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.424+0000 7fe3f96f1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3f40b3e60 0x7fe3f4146dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54298/0 (socket says 172.21.15.43:54298) 2023-06-28T20:08:00.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.424+0000 7fe3f96f1700 1 -- 172.21.15.43:0/1300962281 learned_addr learned my addr 172.21.15.43:0/1300962281 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:00.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.424+0000 7fe3fa6f3700 1 --2- 172.21.15.43:0/1300962281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3f40b0600 0x7fe3f40b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:00.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.424+0000 7fe3f9ef2700 1 --2- 172.21.15.43:0/1300962281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3e0003a00 0x7fe3f40b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:00.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.424+0000 7fe3f96f1700 1 -- 172.21.15.43:0/1300962281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3f40b0600 msgr2=0x7fe3f40b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:00.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.424+0000 7fe3f96f1700 1 --2- 172.21.15.43:0/1300962281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3f40b0600 0x7fe3f40b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.424+0000 7fe3f96f1700 1 -- 172.21.15.43:0/1300962281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3e0003a00 msgr2=0x7fe3f40b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:00.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.424+0000 7fe3f96f1700 1 --2- 172.21.15.43:0/1300962281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3e0003a00 0x7fe3f40b4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.434 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.424+0000 7fe3f96f1700 1 -- 172.21.15.43:0/1300962281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f415cf10 con 0x7fe3f40b3e60 2023-06-28T20:08:00.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.424+0000 7fe3f9ef2700 1 --2- 172.21.15.43:0/1300962281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3e0003a00 0x7fe3f40b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:00.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.425+0000 7fe3f96f1700 1 --2- 172.21.15.43:0/1300962281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3f40b3e60 0x7fe3f4146dd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0011aa0 tx=0x7fe3f0011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:00.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.425+0000 7fe3ea7fc700 1 -- 172.21.15.43:0/1300962281 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f000f940 con 0x7fe3f40b3e60 2023-06-28T20:08:00.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.425+0000 7fe3ea7fc700 1 -- 172.21.15.43:0/1300962281 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f000faa0 con 0x7fe3f40b3e60 2023-06-28T20:08:00.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.425+0000 7fe3fc156700 1 -- 172.21.15.43:0/1300962281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f415d1f0 con 0x7fe3f40b3e60 2023-06-28T20:08:00.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.425+0000 7fe3ea7fc700 1 -- 172.21.15.43:0/1300962281 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f000fe50 con 0x7fe3f40b3e60 2023-06-28T20:08:00.435 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.425+0000 7fe3fc156700 1 -- 172.21.15.43:0/1300962281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f415d7b0 con 0x7fe3f40b3e60 2023-06-28T20:08:00.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.426+0000 7fe3fc156700 1 -- 172.21.15.43:0/1300962281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f4071a50 con 0x7fe3f40b3e60 2023-06-28T20:08:00.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.426+0000 7fe3ea7fc700 1 -- 172.21.15.43:0/1300962281 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe3f0014440 con 0x7fe3f40b3e60 2023-06-28T20:08:00.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.427+0000 7fe3ea7fc700 1 --2- 172.21.15.43:0/1300962281 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe3d806ace0 0x7fe3d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:00.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.427+0000 7fe3ea7fc700 1 -- 172.21.15.43:0/1300962281 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe3f0089bf0 con 0x7fe3f40b3e60 2023-06-28T20:08:00.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.427+0000 7fe3f9ef2700 1 --2- 172.21.15.43:0/1300962281 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe3d806ace0 0x7fe3d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:00.436 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.427+0000 7fe3f9ef2700 1 --2- 172.21.15.43:0/1300962281 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe3d806ace0 0x7fe3d806d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4008fd0 tx=0x7fe3e400a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:00.437 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.431+0000 7fe3ea7fc700 1 -- 172.21.15.43:0/1300962281 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe3f00587f0 con 0x7fe3f40b3e60 2023-06-28T20:08:00.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.628+0000 7fe3fc156700 1 -- 172.21.15.43:0/1300962281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe3f40728b0 con 0x7fe3f40b3e60 2023-06-28T20:08:00.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.629+0000 7fe3ea7fc700 1 -- 172.21.15.43:0/1300962281 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe3f0058380 con 0x7fe3f40b3e60 2023-06-28T20:08:00.632 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-06-28T20:08:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.631+0000 7fe3fc156700 1 -- 172.21.15.43:0/1300962281 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe3d806ace0 msgr2=0x7fe3d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.631+0000 7fe3fc156700 1 --2- 172.21.15.43:0/1300962281 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe3d806ace0 0x7fe3d806d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4008fd0 tx=0x7fe3e400a000 comp rx=0 tx=0).stop 2023-06-28T20:08:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.631+0000 7fe3fc156700 1 -- 172.21.15.43:0/1300962281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3f40b3e60 msgr2=0x7fe3f4146dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:00.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.631+0000 7fe3fc156700 1 --2- 172.21.15.43:0/1300962281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3f40b3e60 0x7fe3f4146dd0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0011aa0 tx=0x7fe3f0011db0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.631+0000 7fe3fc156700 1 -- 172.21.15.43:0/1300962281 shutdown_connections 2023-06-28T20:08:00.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.631+0000 7fe3fc156700 1 --2- 172.21.15.43:0/1300962281 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe3d806ace0 0x7fe3d806d190 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.631+0000 7fe3fc156700 1 --2- 172.21.15.43:0/1300962281 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe3f40b0600 0x7fe3f40b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.631+0000 7fe3fc156700 1 --2- 172.21.15.43:0/1300962281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3e0003a00 0x7fe3f40b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.631+0000 7fe3fc156700 1 --2- 172.21.15.43:0/1300962281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3f40b3e60 0x7fe3f4146dd0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.631+0000 7fe3fc156700 1 -- 172.21.15.43:0/1300962281 >> 172.21.15.43:0/1300962281 conn(0x7fe3f406b000 msgr2=0x7fe3f40b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:00.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.631+0000 7fe3fc156700 1 -- 172.21.15.43:0/1300962281 shutdown_connections 2023-06-28T20:08:00.636 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.631+0000 7fe3fc156700 1 -- 172.21.15.43:0/1300962281 wait complete. 2023-06-28T20:08:00.646 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-28T20:08:00.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.771+0000 7fc17c265700 1 Processor -- start 2023-06-28T20:08:00.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.772+0000 7fc17c265700 1 -- start start 2023-06-28T20:08:00.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.772+0000 7fc17c265700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc1740cfea0 0x7fc17406a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:00.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.772+0000 7fc17c265700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc174069ef0 con 0x7fc1740c2650 2023-06-28T20:08:00.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.772+0000 7fc17c265700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc174164e90 con 0x7fc17406ac10 2023-06-28T20:08:00.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.772+0000 7fc17c265700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc17406a2b0 con 0x7fc1740cfea0 2023-06-28T20:08:00.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.773+0000 7fc17a001700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc1740c2650 0x7fc174164660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43490/0 (socket says 172.21.15.43:43490) 2023-06-28T20:08:00.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.773+0000 7fc17a001700 1 -- 172.21.15.43:0/2881444052 learned_addr learned my addr 172.21.15.43:0/2881444052 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:00.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.773+0000 7fc179800700 1 --2- 172.21.15.43:0/2881444052 >> v2:172.21.15.178:3300/0 conn(0x7fc1740cfea0 0x7fc17406a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:00.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.773+0000 7fc178fff700 1 -- 172.21.15.43:0/2881444052 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 435103356 0 0) 0x7fc174069ef0 con 0x7fc1740c2650 2023-06-28T20:08:00.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.773+0000 7fc178fff700 1 -- 172.21.15.43:0/2881444052 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1600035b0 con 0x7fc1740c2650 2023-06-28T20:08:00.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.773+0000 7fc179800700 1 -- 172.21.15.43:0/2881444052 >> v1:172.21.15.178:6789/0 conn(0x7fc17406ac10 legacy=0x7fc17406c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:00.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.773+0000 7fc179800700 1 -- 172.21.15.43:0/2881444052 >> v1:172.21.15.43:6789/0 conn(0x7fc1740c2650 legacy=0x7fc174164660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:00.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.773+0000 7fc179800700 1 -- 172.21.15.43:0/2881444052 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc174165170 con 0x7fc1740cfea0 2023-06-28T20:08:00.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.773+0000 7fc179800700 1 --2- 172.21.15.43:0/2881444052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1740cfea0 0x7fc17406a6d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc164009840 tx=0x7fc164009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27d5c4fd679955f0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:00.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.774+0000 7fc178fff700 1 -- 172.21.15.43:0/2881444052 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164011830 con 0x7fc1740cfea0 2023-06-28T20:08:00.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.774+0000 7fc178fff700 1 -- 172.21.15.43:0/2881444052 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc164011990 con 0x7fc1740cfea0 2023-06-28T20:08:00.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.774+0000 7fc178fff700 1 -- 172.21.15.43:0/2881444052 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164011d00 con 0x7fc1740cfea0 2023-06-28T20:08:00.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.774+0000 7fc17c265700 1 -- 172.21.15.43:0/2881444052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1740cfea0 msgr2=0x7fc17406a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:00.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.774+0000 7fc17c265700 1 --2- 172.21.15.43:0/2881444052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1740cfea0 0x7fc17406a6d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc164009840 tx=0x7fc164009b50 comp rx=0 tx=0).stop 2023-06-28T20:08:00.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.774+0000 7fc17c265700 1 -- 172.21.15.43:0/2881444052 shutdown_connections 2023-06-28T20:08:00.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.774+0000 7fc17c265700 1 --2- 172.21.15.43:0/2881444052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1740cfea0 0x7fc17406a6d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.774+0000 7fc17c265700 1 -- 172.21.15.43:0/2881444052 >> 172.21.15.43:0/2881444052 conn(0x7fc1740b1a50 msgr2=0x7fc1740cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:00.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.774+0000 7fc17c265700 1 -- 172.21.15.43:0/2881444052 shutdown_connections 2023-06-28T20:08:00.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.774+0000 7fc17c265700 1 -- 172.21.15.43:0/2881444052 wait complete. 2023-06-28T20:08:00.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.776+0000 7fc17c265700 1 Processor -- start 2023-06-28T20:08:00.785 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.776+0000 7fc17c265700 1 -- start start 2023-06-28T20:08:00.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.776+0000 7fc17c265700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17406ac10 0x7fc1741eb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:00.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.776+0000 7fc17c265700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1740c2650 0x7fc174158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:00.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.776+0000 7fc17c265700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1740cfea0 0x7fc1741eca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:00.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.776+0000 7fc17c265700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17415df30 con 0x7fc1740cfea0 2023-06-28T20:08:00.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.776+0000 7fc17c265700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1741ea460 con 0x7fc17406ac10 2023-06-28T20:08:00.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.776+0000 7fc17c265700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1741ea6b0 con 0x7fc1740c2650 2023-06-28T20:08:00.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.777+0000 7fc17a802700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1740cfea0 0x7fc1741eca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:00.786 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.777+0000 7fc17a802700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1740cfea0 0x7fc1741eca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54314/0 (socket says 172.21.15.43:54314) 2023-06-28T20:08:00.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.777+0000 7fc17a802700 1 -- 172.21.15.43:0/4218392853 learned_addr learned my addr 172.21.15.43:0/4218392853 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:00.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.777+0000 7fc17a001700 1 --2- 172.21.15.43:0/4218392853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17406ac10 0x7fc1741eb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:00.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.777+0000 7fc179800700 1 --2- 172.21.15.43:0/4218392853 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1740c2650 0x7fc174158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:00.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.777+0000 7fc17a802700 1 -- 172.21.15.43:0/4218392853 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1740c2650 msgr2=0x7fc174158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:00.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.777+0000 7fc17a802700 1 --2- 172.21.15.43:0/4218392853 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1740c2650 0x7fc174158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.777+0000 7fc17a802700 1 -- 172.21.15.43:0/4218392853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17406ac10 msgr2=0x7fc1741eb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:00.787 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.777+0000 7fc17a802700 1 --2- 172.21.15.43:0/4218392853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17406ac10 0x7fc1741eb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.777+0000 7fc17a802700 1 -- 172.21.15.43:0/4218392853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1640094a0 con 0x7fc1740cfea0 2023-06-28T20:08:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.777+0000 7fc179800700 1 --2- 172.21.15.43:0/4218392853 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1740c2650 0x7fc174158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.777+0000 7fc17a001700 1 --2- 172.21.15.43:0/4218392853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17406ac10 0x7fc1741eb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.777+0000 7fc17a802700 1 --2- 172.21.15.43:0/4218392853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1740cfea0 0x7fc1741eca30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc16c00cbc0 tx=0x7fc16c00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.777+0000 7fc16a7fc700 1 -- 172.21.15.43:0/4218392853 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c003c00 con 0x7fc1740cfea0 2023-06-28T20:08:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.777+0000 7fc16a7fc700 1 -- 172.21.15.43:0/4218392853 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16c003d60 con 0x7fc1740cfea0 2023-06-28T20:08:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.777+0000 7fc17c265700 1 -- 172.21.15.43:0/4218392853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1741ea990 con 0x7fc1740cfea0 2023-06-28T20:08:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.777+0000 7fc16a7fc700 1 -- 172.21.15.43:0/4218392853 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c00ab30 con 0x7fc1740cfea0 2023-06-28T20:08:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.778+0000 7fc17c265700 1 -- 172.21.15.43:0/4218392853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1741ecf70 con 0x7fc1740cfea0 2023-06-28T20:08:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.778+0000 7fc17c265700 1 -- 172.21.15.43:0/4218392853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1741ed1c0 con 0x7fc1740cfea0 2023-06-28T20:08:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.779+0000 7fc16a7fc700 1 -- 172.21.15.43:0/4218392853 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc16c00ac90 con 0x7fc1740cfea0 2023-06-28T20:08:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.779+0000 7fc16a7fc700 1 --2- 172.21.15.43:0/4218392853 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc15806ace0 0x7fc15806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.779+0000 7fc16a7fc700 1 -- 172.21.15.43:0/4218392853 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc16c0894e0 con 0x7fc1740cfea0 2023-06-28T20:08:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.779+0000 7fc17a001700 1 --2- 172.21.15.43:0/4218392853 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc15806ace0 0x7fc15806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:00.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.780+0000 7fc17a001700 1 --2- 172.21.15.43:0/4218392853 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc15806ace0 0x7fc15806d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc1700011e0 tx=0x7fc170009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:00.790 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.783+0000 7fc16a7fc700 1 -- 172.21.15.43:0/4218392853 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc16c054590 con 0x7fc1740cfea0 2023-06-28T20:08:00.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.982+0000 7fc17c265700 1 -- 172.21.15.43:0/4218392853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc17415e070 con 0x7fc1740cfea0 2023-06-28T20:08:00.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.982+0000 7fc16a7fc700 1 -- 172.21.15.43:0/4218392853 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc17415e070 con 0x7fc1740cfea0 2023-06-28T20:08:00.986 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-06-28T20:08:00.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.984+0000 7fc17c265700 1 -- 172.21.15.43:0/4218392853 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc15806ace0 msgr2=0x7fc15806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:00.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.984+0000 7fc17c265700 1 --2- 172.21.15.43:0/4218392853 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc15806ace0 0x7fc15806d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc1700011e0 tx=0x7fc170009040 comp rx=0 tx=0).stop 2023-06-28T20:08:00.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.984+0000 7fc17c265700 1 -- 172.21.15.43:0/4218392853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1740cfea0 msgr2=0x7fc1741eca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:00.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.984+0000 7fc17c265700 1 --2- 172.21.15.43:0/4218392853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1740cfea0 0x7fc1741eca30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc16c00cbc0 tx=0x7fc16c00cf80 comp rx=0 tx=0).stop 2023-06-28T20:08:00.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.984+0000 7fc17c265700 1 -- 172.21.15.43:0/4218392853 shutdown_connections 2023-06-28T20:08:00.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.984+0000 7fc17c265700 1 --2- 172.21.15.43:0/4218392853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1740cfea0 0x7fc1741eca30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.984+0000 7fc17c265700 1 --2- 172.21.15.43:0/4218392853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17406ac10 0x7fc1741eb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.984+0000 7fc17c265700 1 --2- 172.21.15.43:0/4218392853 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc15806ace0 0x7fc15806d190 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.984+0000 7fc17c265700 1 --2- 172.21.15.43:0/4218392853 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc1740c2650 0x7fc174158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:00.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.984+0000 7fc17c265700 1 -- 172.21.15.43:0/4218392853 >> 172.21.15.43:0/4218392853 conn(0x7fc1740b1a50 msgr2=0x7fc174070500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:00.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.984+0000 7fc17c265700 1 -- 172.21.15.43:0/4218392853 shutdown_connections 2023-06-28T20:08:00.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:00.984+0000 7fc17c265700 1 -- 172.21.15.43:0/4218392853 wait complete. 2023-06-28T20:08:01.000 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-28T20:08:01.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.124+0000 7fac3af74700 1 Processor -- start 2023-06-28T20:08:01.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.124+0000 7fac3af74700 1 -- start start 2023-06-28T20:08:01.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.125+0000 7fac3af74700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fac340caff0 0x7fac3406b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:01.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.125+0000 7fac3af74700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac3406a020 con 0x7fac3406c7b0 2023-06-28T20:08:01.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.125+0000 7fac3af74700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac3406a300 con 0x7fac340cb3c0 2023-06-28T20:08:01.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.125+0000 7fac3af74700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac3406a5e0 con 0x7fac340caff0 2023-06-28T20:08:01.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.125+0000 7fac39511700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fac3406c7b0 0x7fac34071060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43508/0 (socket says 172.21.15.43:43508) 2023-06-28T20:08:01.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.125+0000 7fac39511700 1 -- 172.21.15.43:0/1684189552 learned_addr learned my addr 172.21.15.43:0/1684189552 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:01.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.126+0000 7fac38d10700 1 --2- 172.21.15.43:0/1684189552 >> v2:172.21.15.178:3300/0 conn(0x7fac340caff0 0x7fac3406b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:01.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.126+0000 7fac337fe700 1 -- 172.21.15.43:0/1684189552 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278657259 0 0) 0x7fac3406a020 con 0x7fac3406c7b0 2023-06-28T20:08:01.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.126+0000 7fac337fe700 1 -- 172.21.15.43:0/1684189552 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac240035b0 con 0x7fac3406c7b0 2023-06-28T20:08:01.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.126+0000 7fac337fe700 1 -- 172.21.15.43:0/1684189552 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1084441486 0 0) 0x7fac240035b0 con 0x7fac3406c7b0 2023-06-28T20:08:01.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.126+0000 7fac337fe700 1 -- 172.21.15.43:0/1684189552 >> v2:172.21.15.178:3300/0 conn(0x7fac340caff0 msgr2=0x7fac3406b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:01.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.126+0000 7fac337fe700 1 --2- 172.21.15.43:0/1684189552 >> v2:172.21.15.178:3300/0 conn(0x7fac340caff0 0x7fac3406b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:01.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.126+0000 7fac337fe700 1 -- 172.21.15.43:0/1684189552 >> v1:172.21.15.178:6789/0 conn(0x7fac340cb3c0 legacy=0x7fac3406c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:01.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.126+0000 7fac337fe700 1 -- 172.21.15.43:0/1684189552 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac3406a8c0 con 0x7fac3406c7b0 2023-06-28T20:08:01.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.126+0000 7fac337fe700 1 -- 172.21.15.43:0/1684189552 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7fac2c0019f0 con 0x7fac3406c7b0 2023-06-28T20:08:01.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.126+0000 7fac337fe700 1 -- 172.21.15.43:0/1684189552 >> v1:172.21.15.43:6789/0 conn(0x7fac3406c7b0 legacy=0x7fac34071060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:01.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.126+0000 7fac337fe700 1 --2- 172.21.15.43:0/1684189552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac24003a00 0x7fac24003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:01.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.126+0000 7fac337fe700 1 --2- 172.21.15.43:0/1684189552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac24004850 0x7fac24004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:01.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.126+0000 7fac337fe700 1 --2- 172.21.15.43:0/1684189552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac24005200 0x7fac2400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:01.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.126+0000 7fac337fe700 1 -- 172.21.15.43:0/1684189552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3406a5e0 con 0x7fac24004850 2023-06-28T20:08:01.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.126+0000 7fac337fe700 1 -- 172.21.15.43:0/1684189552 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2400c110 con 0x7fac24005200 2023-06-28T20:08:01.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.126+0000 7fac337fe700 1 -- 172.21.15.43:0/1684189552 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2400c2d0 con 0x7fac24003a00 2023-06-28T20:08:01.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.126+0000 7fac33fff700 1 --2- 172.21.15.43:0/1684189552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac24004850 0x7fac24004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:01.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.126+0000 7fac38d10700 1 --2- 172.21.15.43:0/1684189552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac24003a00 0x7fac24003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:01.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.126+0000 7fac39511700 1 --2- 172.21.15.43:0/1684189552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac24005200 0x7fac2400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:01.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.127+0000 7fac33fff700 1 -- 172.21.15.43:0/1684189552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac24003a00 msgr2=0x7fac24003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:01.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.127+0000 7fac33fff700 1 --2- 172.21.15.43:0/1684189552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac24003a00 0x7fac24003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:01.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.127+0000 7fac33fff700 1 -- 172.21.15.43:0/1684189552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac24005200 msgr2=0x7fac2400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:01.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.127+0000 7fac33fff700 1 --2- 172.21.15.43:0/1684189552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac24005200 0x7fac2400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:01.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.127+0000 7fac33fff700 1 -- 172.21.15.43:0/1684189552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac240035b0 con 0x7fac24004850 2023-06-28T20:08:01.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.127+0000 7fac39511700 1 --2- 172.21.15.43:0/1684189552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac24005200 0x7fac2400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:01.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.127+0000 7fac39511700 1 -- 172.21.15.43:0/1684189552 reap_dead start 2023-06-28T20:08:01.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.127+0000 7fac33fff700 1 --2- 172.21.15.43:0/1684189552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac24004850 0x7fac24004cc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fac340cb800 tx=0x7fac24005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1c3823fce6169f server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.127+0000 7fac337fe700 1 -- 172.21.15.43:0/1684189552 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac200052d0 con 0x7fac24004850 2023-06-28T20:08:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.127+0000 7fac337fe700 1 -- 172.21.15.43:0/1684189552 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac20005430 con 0x7fac24004850 2023-06-28T20:08:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.127+0000 7fac337fe700 1 -- 172.21.15.43:0/1684189552 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac200038c0 con 0x7fac24004850 2023-06-28T20:08:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.127+0000 7fac3af74700 1 -- 172.21.15.43:0/1684189552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac24004850 msgr2=0x7fac24004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.127+0000 7fac3af74700 1 --2- 172.21.15.43:0/1684189552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac24004850 0x7fac24004cc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fac340cb800 tx=0x7fac24005880 comp rx=0 tx=0).stop 2023-06-28T20:08:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.127+0000 7fac3af74700 1 -- 172.21.15.43:0/1684189552 shutdown_connections 2023-06-28T20:08:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.127+0000 7fac3af74700 1 --2- 172.21.15.43:0/1684189552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac24004850 0x7fac24004cc0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:01.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.127+0000 7fac3af74700 1 -- 172.21.15.43:0/1684189552 >> 172.21.15.43:0/1684189552 conn(0x7fac340b1a50 msgr2=0x7fac340cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:01.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.127+0000 7fac3af74700 1 -- 172.21.15.43:0/1684189552 shutdown_connections 2023-06-28T20:08:01.139 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.128+0000 7fac3af74700 1 -- 172.21.15.43:0/1684189552 wait complete. 2023-06-28T20:08:01.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.129+0000 7fac3af74700 1 Processor -- start 2023-06-28T20:08:01.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.129+0000 7fac3af74700 1 -- start start 2023-06-28T20:08:01.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.130+0000 7fac3af74700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac24004850 0x7fac340b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:01.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.130+0000 7fac3af74700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac340ac870 0x7fac340b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:01.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.130+0000 7fac3af74700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac340b2d00 0x7fac340cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:01.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.130+0000 7fac3af74700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac340cdac0 con 0x7fac340ac870 2023-06-28T20:08:01.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.130+0000 7fac3af74700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac340ccf20 con 0x7fac24004850 2023-06-28T20:08:01.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.130+0000 7fac3af74700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac340cd190 con 0x7fac340b2d00 2023-06-28T20:08:01.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.130+0000 7fac33fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac340ac870 0x7fac340b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:01.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.130+0000 7fac33fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac340ac870 0x7fac340b48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54332/0 (socket says 172.21.15.43:54332) 2023-06-28T20:08:01.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.130+0000 7fac33fff700 1 -- 172.21.15.43:0/2649227599 learned_addr learned my addr 172.21.15.43:0/2649227599 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:01.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.130+0000 7fac38d10700 1 --2- 172.21.15.43:0/2649227599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac24004850 0x7fac340b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.130+0000 7fac39511700 1 --2- 172.21.15.43:0/2649227599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac340b2d00 0x7fac340cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.130+0000 7fac33fff700 1 -- 172.21.15.43:0/2649227599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac340b2d00 msgr2=0x7fac340cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.130+0000 7fac33fff700 1 --2- 172.21.15.43:0/2649227599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac340b2d00 0x7fac340cdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.130+0000 7fac33fff700 1 -- 172.21.15.43:0/2649227599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac24004850 msgr2=0x7fac340b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.130+0000 7fac33fff700 1 --2- 172.21.15.43:0/2649227599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac24004850 0x7fac340b0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.130+0000 7fac33fff700 1 -- 172.21.15.43:0/2649227599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac340cd470 con 0x7fac340ac870 2023-06-28T20:08:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.130+0000 7fac39511700 1 --2- 172.21.15.43:0/2649227599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac340b2d00 0x7fac340cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:01.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.130+0000 7fac33fff700 1 --2- 172.21.15.43:0/2649227599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac340ac870 0x7fac340b48f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fac20015040 tx=0x7fac20001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:01.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.130+0000 7fac317fa700 1 -- 172.21.15.43:0/2649227599 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac20001290 con 0x7fac340ac870 2023-06-28T20:08:01.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.130+0000 7fac317fa700 1 -- 172.21.15.43:0/2649227599 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2001d480 con 0x7fac340ac870 2023-06-28T20:08:01.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.130+0000 7fac317fa700 1 -- 172.21.15.43:0/2649227599 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2001d870 con 0x7fac340ac870 2023-06-28T20:08:01.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.131+0000 7fac3af74700 1 -- 172.21.15.43:0/2649227599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac340cd750 con 0x7fac340ac870 2023-06-28T20:08:01.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.131+0000 7fac3af74700 1 -- 172.21.15.43:0/2649227599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac340b5010 con 0x7fac340ac870 2023-06-28T20:08:01.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.131+0000 7fac3af74700 1 -- 172.21.15.43:0/2649227599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac340cdc70 con 0x7fac340ac870 2023-06-28T20:08:01.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.133+0000 7fac317fa700 1 -- 172.21.15.43:0/2649227599 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fac20019040 con 0x7fac340ac870 2023-06-28T20:08:01.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.133+0000 7fac317fa700 1 --2- 172.21.15.43:0/2649227599 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fac1806ad10 0x7fac1806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:01.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.133+0000 7fac317fa700 1 -- 172.21.15.43:0/2649227599 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fac2008a100 con 0x7fac340ac870 2023-06-28T20:08:01.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.133+0000 7fac38d10700 1 --2- 172.21.15.43:0/2649227599 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fac1806ad10 0x7fac1806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:01.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.133+0000 7fac38d10700 1 --2- 172.21.15.43:0/2649227599 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fac1806ad10 0x7fac1806d1c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fac28009c20 tx=0x7fac28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:01.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.136+0000 7fac317fa700 1 -- 172.21.15.43:0/2649227599 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fac20055320 con 0x7fac340ac870 2023-06-28T20:08:01.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.338+0000 7fac3af74700 1 -- 172.21.15.43:0/2649227599 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fac340728b0 con 0x7fac340ac870 2023-06-28T20:08:01.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.339+0000 7fac317fa700 1 -- 172.21.15.43:0/2649227599 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fac20058940 con 0x7fac340ac870 2023-06-28T20:08:01.343 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-06-28T20:08:01.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.341+0000 7fac3af74700 1 -- 172.21.15.43:0/2649227599 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fac1806ad10 msgr2=0x7fac1806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:01.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.341+0000 7fac3af74700 1 --2- 172.21.15.43:0/2649227599 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fac1806ad10 0x7fac1806d1c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fac28009c20 tx=0x7fac28008040 comp rx=0 tx=0).stop 2023-06-28T20:08:01.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.341+0000 7fac3af74700 1 -- 172.21.15.43:0/2649227599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac340ac870 msgr2=0x7fac340b48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:01.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.341+0000 7fac3af74700 1 --2- 172.21.15.43:0/2649227599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac340ac870 0x7fac340b48f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fac20015040 tx=0x7fac20001120 comp rx=0 tx=0).stop 2023-06-28T20:08:01.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.341+0000 7fac3af74700 1 -- 172.21.15.43:0/2649227599 shutdown_connections 2023-06-28T20:08:01.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.341+0000 7fac3af74700 1 --2- 172.21.15.43:0/2649227599 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fac1806ad10 0x7fac1806d1c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:01.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.341+0000 7fac3af74700 1 --2- 172.21.15.43:0/2649227599 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fac340b2d00 0x7fac340cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:01.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.341+0000 7fac3af74700 1 --2- 172.21.15.43:0/2649227599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac24004850 0x7fac340b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:01.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.341+0000 7fac3af74700 1 --2- 172.21.15.43:0/2649227599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac340ac870 0x7fac340b48f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:01.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.341+0000 7fac3af74700 1 -- 172.21.15.43:0/2649227599 >> 172.21.15.43:0/2649227599 conn(0x7fac340b1a50 msgr2=0x7fac341591a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:01.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.341+0000 7fac3af74700 1 -- 172.21.15.43:0/2649227599 shutdown_connections 2023-06-28T20:08:01.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.341+0000 7fac3af74700 1 -- 172.21.15.43:0/2649227599 wait complete. 2023-06-28T20:08:01.357 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-28T20:08:01.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.482+0000 7fd426ca5700 1 Processor -- start 2023-06-28T20:08:01.486 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.482+0000 7fd426ca5700 1 -- start start 2023-06-28T20:08:01.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.483+0000 7fd426ca5700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd420160e30 0x7fd4200b1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:01.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.483+0000 7fd426ca5700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4200b1e80 0x7fd4200ce540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:01.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.483+0000 7fd426ca5700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4200b22a0 con 0x7fd42015c6a0 2023-06-28T20:08:01.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.483+0000 7fd426ca5700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd42015ca70 con 0x7fd420160e30 2023-06-28T20:08:01.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.483+0000 7fd426ca5700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd420071440 con 0x7fd4200b1e80 2023-06-28T20:08:01.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.483+0000 7fd41ffff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd420160e30 0x7fd4200b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:01.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.483+0000 7fd425242700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4200b1e80 0x7fd4200ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:01.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.483+0000 7fd424a41700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fd42015c6a0 0x7fd42006d4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:42648/0 (socket says 172.21.15.43:42648) 2023-06-28T20:08:01.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.483+0000 7fd41ffff700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd420160e30 0x7fd4200b1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:40404/0 (socket says 172.21.15.43:40404) 2023-06-28T20:08:01.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.483+0000 7fd425242700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4200b1e80 0x7fd4200ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.43:38314/0 (socket says 172.21.15.43:38314) 2023-06-28T20:08:01.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.483+0000 7fd424a41700 1 -- 172.21.15.43:0/4044649296 learned_addr learned my addr 172.21.15.43:0/4044649296 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:01.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.483+0000 7fd41ffff700 1 -- 172.21.15.43:0/4044649296 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4200b1e80 msgr2=0x7fd4200ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:01.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.483+0000 7fd41ffff700 1 --2- 172.21.15.43:0/4044649296 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4200b1e80 0x7fd4200ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:01.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.483+0000 7fd41ffff700 1 -- 172.21.15.43:0/4044649296 >> v1:172.21.15.178:6789/0 conn(0x7fd42015c6a0 legacy=0x7fd42006d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:01.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.483+0000 7fd41ffff700 1 -- 172.21.15.43:0/4044649296 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd420071720 con 0x7fd420160e30 2023-06-28T20:08:01.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.484+0000 7fd41ffff700 1 --2- 172.21.15.43:0/4044649296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd420160e30 0x7fd4200b1940 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009840 tx=0x7fd40c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc5d5f66dad2aae4 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:01.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.484+0000 7fd41f7fe700 1 -- 172.21.15.43:0/4044649296 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c011830 con 0x7fd420160e30 2023-06-28T20:08:01.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.484+0000 7fd41f7fe700 1 -- 172.21.15.43:0/4044649296 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40c011990 con 0x7fd420160e30 2023-06-28T20:08:01.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.484+0000 7fd41f7fe700 1 -- 172.21.15.43:0/4044649296 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c011d20 con 0x7fd420160e30 2023-06-28T20:08:01.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.484+0000 7fd426ca5700 1 -- 172.21.15.43:0/4044649296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd420160e30 msgr2=0x7fd4200b1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:01.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.484+0000 7fd426ca5700 1 --2- 172.21.15.43:0/4044649296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd420160e30 0x7fd4200b1940 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd40c009840 tx=0x7fd40c009b50 comp rx=0 tx=0).stop 2023-06-28T20:08:01.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.484+0000 7fd426ca5700 1 -- 172.21.15.43:0/4044649296 shutdown_connections 2023-06-28T20:08:01.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.484+0000 7fd426ca5700 1 --2- 172.21.15.43:0/4044649296 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4200b1e80 0x7fd4200ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:01.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.484+0000 7fd426ca5700 1 --2- 172.21.15.43:0/4044649296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd420160e30 0x7fd4200b1940 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:01.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.484+0000 7fd426ca5700 1 -- 172.21.15.43:0/4044649296 >> 172.21.15.43:0/4044649296 conn(0x7fd42006b000 msgr2=0x7fd42006b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:01.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.484+0000 7fd426ca5700 1 -- 172.21.15.43:0/4044649296 shutdown_connections 2023-06-28T20:08:01.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.484+0000 7fd426ca5700 1 -- 172.21.15.43:0/4044649296 wait complete. 2023-06-28T20:08:01.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.486+0000 7fd426ca5700 1 Processor -- start 2023-06-28T20:08:01.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.486+0000 7fd426ca5700 1 -- start start 2023-06-28T20:08:01.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.487+0000 7fd426ca5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4200b1e80 0x7fd4200c3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:01.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.487+0000 7fd426ca5700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd42015c6a0 0x7fd420155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:01.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.487+0000 7fd426ca5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd420160e30 0x7fd4200ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:01.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.487+0000 7fd426ca5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42015eff0 con 0x7fd4200b1e80 2023-06-28T20:08:01.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.487+0000 7fd426ca5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42015f2d0 con 0x7fd420160e30 2023-06-28T20:08:01.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.487+0000 7fd426ca5700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42015f5b0 con 0x7fd42015c6a0 2023-06-28T20:08:01.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.487+0000 7fd424a41700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4200b1e80 0x7fd4200c3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:01.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.487+0000 7fd424a41700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4200b1e80 0x7fd4200c3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54344/0 (socket says 172.21.15.43:54344) 2023-06-28T20:08:01.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.487+0000 7fd424a41700 1 -- 172.21.15.43:0/3108639408 learned_addr learned my addr 172.21.15.43:0/3108639408 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:01.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.487+0000 7fd425242700 1 --2- 172.21.15.43:0/3108639408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd420160e30 0x7fd4200ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:01.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.487+0000 7fd41ffff700 1 --2- 172.21.15.43:0/3108639408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd42015c6a0 0x7fd420155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:01.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.487+0000 7fd424a41700 1 -- 172.21.15.43:0/3108639408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd42015c6a0 msgr2=0x7fd420155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:01.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.487+0000 7fd424a41700 1 --2- 172.21.15.43:0/3108639408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd42015c6a0 0x7fd420155ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:01.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.487+0000 7fd424a41700 1 -- 172.21.15.43:0/3108639408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd420160e30 msgr2=0x7fd4200ae6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:01.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.487+0000 7fd424a41700 1 --2- 172.21.15.43:0/3108639408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd420160e30 0x7fd4200ae6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.487+0000 7fd424a41700 1 -- 172.21.15.43:0/3108639408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c0094a0 con 0x7fd4200b1e80 2023-06-28T20:08:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.487+0000 7fd424a41700 1 --2- 172.21.15.43:0/3108639408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4200b1e80 0x7fd4200c3620 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd41400c490 tx=0x7fd41400c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.487+0000 7fd41d7fa700 1 -- 172.21.15.43:0/3108639408 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41400cea0 con 0x7fd4200b1e80 2023-06-28T20:08:01.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.487+0000 7fd41d7fa700 1 -- 172.21.15.43:0/3108639408 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd414005d70 con 0x7fd4200b1e80 2023-06-28T20:08:01.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.487+0000 7fd426ca5700 1 -- 172.21.15.43:0/3108639408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd42015f890 con 0x7fd4200b1e80 2023-06-28T20:08:01.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.487+0000 7fd41d7fa700 1 -- 172.21.15.43:0/3108639408 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4140072f0 con 0x7fd4200b1e80 2023-06-28T20:08:01.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.488+0000 7fd426ca5700 1 -- 172.21.15.43:0/3108639408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd420156530 con 0x7fd4200b1e80 2023-06-28T20:08:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.488+0000 7fd426ca5700 1 -- 172.21.15.43:0/3108639408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4200c6e00 con 0x7fd4200b1e80 2023-06-28T20:08:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.489+0000 7fd41d7fa700 1 -- 172.21.15.43:0/3108639408 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd414007450 con 0x7fd4200b1e80 2023-06-28T20:08:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.489+0000 7fd41d7fa700 1 --2- 172.21.15.43:0/3108639408 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd40406ace0 0x7fd40406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.489+0000 7fd41d7fa700 1 -- 172.21.15.43:0/3108639408 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd414088800 con 0x7fd4200b1e80 2023-06-28T20:08:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.489+0000 7fd41ffff700 1 --2- 172.21.15.43:0/3108639408 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd40406ace0 0x7fd40406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:01.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.490+0000 7fd41ffff700 1 --2- 172.21.15.43:0/3108639408 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd40406ace0 0x7fd40406d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd420161270 tx=0x7fd40c01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:01.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.494+0000 7fd41d7fa700 1 -- 172.21.15.43:0/3108639408 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd4200c6e00 con 0x7fd4200b1e80 2023-06-28T20:08:01.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.707+0000 7fd426ca5700 1 -- 172.21.15.43:0/3108639408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd4200c6e00 con 0x7fd4200b1e80 2023-06-28T20:08:01.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.707+0000 7fd41d7fa700 1 -- 172.21.15.43:0/3108639408 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd4200c6e00 con 0x7fd4200b1e80 2023-06-28T20:08:01.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.709+0000 7fd426ca5700 1 -- 172.21.15.43:0/3108639408 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd40406ace0 msgr2=0x7fd40406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:01.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.709+0000 7fd426ca5700 1 --2- 172.21.15.43:0/3108639408 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd40406ace0 0x7fd40406d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd420161270 tx=0x7fd40c01c040 comp rx=0 tx=0).stop 2023-06-28T20:08:01.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.709+0000 7fd426ca5700 1 -- 172.21.15.43:0/3108639408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4200b1e80 msgr2=0x7fd4200c3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:01.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.709+0000 7fd426ca5700 1 --2- 172.21.15.43:0/3108639408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4200b1e80 0x7fd4200c3620 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd41400c490 tx=0x7fd41400c7a0 comp rx=0 tx=0).stop 2023-06-28T20:08:01.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.710+0000 7fd426ca5700 1 -- 172.21.15.43:0/3108639408 shutdown_connections 2023-06-28T20:08:01.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.710+0000 7fd426ca5700 1 --2- 172.21.15.43:0/3108639408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd420160e30 0x7fd4200ae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:01.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.710+0000 7fd426ca5700 1 --2- 172.21.15.43:0/3108639408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4200b1e80 0x7fd4200c3620 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:01.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.710+0000 7fd426ca5700 1 --2- 172.21.15.43:0/3108639408 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd40406ace0 0x7fd40406d190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:01.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.710+0000 7fd426ca5700 1 --2- 172.21.15.43:0/3108639408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd42015c6a0 0x7fd420155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:01.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.710+0000 7fd426ca5700 1 -- 172.21.15.43:0/3108639408 >> 172.21.15.43:0/3108639408 conn(0x7fd42006b000 msgr2=0x7fd4200af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:01.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.710+0000 7fd426ca5700 1 -- 172.21.15.43:0/3108639408 shutdown_connections 2023-06-28T20:08:01.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:01.710+0000 7fd426ca5700 1 -- 172.21.15.43:0/3108639408 wait complete. 2023-06-28T20:08:01.716 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2023-06-28T20:08:01.726 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-06-28T20:08:01.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:01.726 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-06-28T20:08:01.763 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-06-28T20:08:01.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:01.764 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-06-28T20:08:01.821 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:08:01.821 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:08:01.842 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:08:01.843 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:08:01.902 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:08:01.903 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:08:01.963 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:08:02.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.084+0000 7f68a5492700 1 Processor -- start 2023-06-28T20:08:02.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.084+0000 7f68a5492700 1 -- start start 2023-06-28T20:08:02.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.085+0000 7f68a5492700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f68a00af060 0x7f68a00cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:02.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.085+0000 7f68a5492700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68a006ca80 0x7f68a0069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:02.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.085+0000 7f68a5492700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68a0071410 con 0x7f68a006ce80 2023-06-28T20:08:02.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.085+0000 7f68a5492700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68a00716b0 con 0x7f68a00af060 2023-06-28T20:08:02.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.085+0000 7f68a5492700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a0071990 con 0x7f68a006ca80 2023-06-28T20:08:02.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.085+0000 7f689f7fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f68a006ce80 0x7f68a00ac870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43542/0 (socket says 172.21.15.43:43542) 2023-06-28T20:08:02.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.085+0000 7f689f7fe700 1 -- 172.21.15.43:0/4018434053 learned_addr learned my addr 172.21.15.43:0/4018434053 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:02.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.085+0000 7f689effd700 1 --2- 172.21.15.43:0/4018434053 >> v2:172.21.15.178:3300/0 conn(0x7f68a00af060 0x7f68a00cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:02.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.085+0000 7f689e7fc700 1 --2- 172.21.15.43:0/4018434053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68a006ca80 0x7f68a0069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:02.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689dffb700 1 -- 172.21.15.43:0/4018434053 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2417787223 0 0) 0x7f68a0071410 con 0x7f68a006ce80 2023-06-28T20:08:02.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689dffb700 1 -- 172.21.15.43:0/4018434053 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68880035b0 con 0x7f68a006ce80 2023-06-28T20:08:02.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689dffb700 1 -- 172.21.15.43:0/4018434053 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1134485264 0 0) 0x7f68880035b0 con 0x7f68a006ce80 2023-06-28T20:08:02.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689dffb700 1 -- 172.21.15.43:0/4018434053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68a006ca80 msgr2=0x7f68a0069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:02.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689dffb700 1 --2- 172.21.15.43:0/4018434053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68a006ca80 0x7f68a0069ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689dffb700 1 -- 172.21.15.43:0/4018434053 >> v2:172.21.15.178:3300/0 conn(0x7f68a00af060 msgr2=0x7f68a00cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:02.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689dffb700 1 --2- 172.21.15.43:0/4018434053 >> v2:172.21.15.178:3300/0 conn(0x7f68a00af060 0x7f68a00cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689dffb700 1 -- 172.21.15.43:0/4018434053 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a00af430 con 0x7f68a006ce80 2023-06-28T20:08:02.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689dffb700 1 -- 172.21.15.43:0/4018434053 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f68980019f0 con 0x7f68a006ce80 2023-06-28T20:08:02.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689dffb700 1 -- 172.21.15.43:0/4018434053 >> v1:172.21.15.43:6789/0 conn(0x7f68a006ce80 legacy=0x7f68a00ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:02.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689dffb700 1 --2- 172.21.15.43:0/4018434053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6888003a00 0x7f6888003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:02.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689dffb700 1 --2- 172.21.15.43:0/4018434053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6888004430 0x7f68880048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:02.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689dffb700 1 --2- 172.21.15.43:0/4018434053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6888004fb0 0x7f688800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:02.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689dffb700 1 -- 172.21.15.43:0/4018434053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a00716b0 con 0x7f6888004fb0 2023-06-28T20:08:02.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689dffb700 1 -- 172.21.15.43:0/4018434053 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a0071990 con 0x7f6888003a00 2023-06-28T20:08:02.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689dffb700 1 -- 172.21.15.43:0/4018434053 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688800bf50 con 0x7f6888004430 2023-06-28T20:08:02.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689e7fc700 1 --2- 172.21.15.43:0/4018434053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68a006ca80 0x7f68a0069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:08:02.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689effd700 1 --2- 172.21.15.43:0/4018434053 >> v2:172.21.15.178:3300/0 conn(0x7f68a00af060 0x7f68a00cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:08:02.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689f7fe700 1 --2- 172.21.15.43:0/4018434053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6888004fb0 0x7f688800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:02.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689effd700 1 --2- 172.21.15.43:0/4018434053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6888003a00 0x7f6888003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:02.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689e7fc700 1 --2- 172.21.15.43:0/4018434053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6888004430 0x7f68880048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:02.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689f7fe700 1 -- 172.21.15.43:0/4018434053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6888004430 msgr2=0x7f68880048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:02.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689f7fe700 1 --2- 172.21.15.43:0/4018434053 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6888004430 0x7f68880048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689f7fe700 1 -- 172.21.15.43:0/4018434053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6888003a00 msgr2=0x7f6888003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:02.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689f7fe700 1 --2- 172.21.15.43:0/4018434053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6888003a00 0x7f6888003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689f7fe700 1 -- 172.21.15.43:0/4018434053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68880035b0 con 0x7f6888004fb0 2023-06-28T20:08:02.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.086+0000 7f689effd700 1 --2- 172.21.15.43:0/4018434053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6888003a00 0x7f6888003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:02.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.087+0000 7f689f7fe700 1 --2- 172.21.15.43:0/4018434053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6888004fb0 0x7f688800ba10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f689800a860 tx=0x7f6888005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94bcd5b89ff594f1 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:02.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.088+0000 7f689dffb700 1 -- 172.21.15.43:0/4018434053 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6898012600 con 0x7f6888004fb0 2023-06-28T20:08:02.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.088+0000 7f689dffb700 1 -- 172.21.15.43:0/4018434053 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6898012780 con 0x7f6888004fb0 2023-06-28T20:08:02.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.088+0000 7f689dffb700 1 -- 172.21.15.43:0/4018434053 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6898012b30 con 0x7f6888004fb0 2023-06-28T20:08:02.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.089+0000 7f68a5492700 1 -- 172.21.15.43:0/4018434053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6888004fb0 msgr2=0x7f688800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:02.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.089+0000 7f68a5492700 1 --2- 172.21.15.43:0/4018434053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6888004fb0 0x7f688800ba10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f689800a860 tx=0x7f6888005880 comp rx=0 tx=0).stop 2023-06-28T20:08:02.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.089+0000 7f689f7fe700 1 -- 172.21.15.43:0/4018434053 reap_dead start 2023-06-28T20:08:02.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.089+0000 7f68a5492700 1 -- 172.21.15.43:0/4018434053 shutdown_connections 2023-06-28T20:08:02.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.089+0000 7f68a5492700 1 -- 172.21.15.43:0/4018434053 >> 172.21.15.43:0/4018434053 conn(0x7f68a00b1720 msgr2=0x7f68a00b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:02.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.089+0000 7f68a5492700 1 -- 172.21.15.43:0/4018434053 shutdown_connections 2023-06-28T20:08:02.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.089+0000 7f68a5492700 1 -- 172.21.15.43:0/4018434053 wait complete. 2023-06-28T20:08:02.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.090+0000 7f68a5492700 1 Processor -- start 2023-06-28T20:08:02.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.091+0000 7f68a5492700 1 -- start start 2023-06-28T20:08:02.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.091+0000 7f68a5492700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68a014fa30 0x7f68a00c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:02.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.091+0000 7f68a5492700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a00c4610 0x7f68a0158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:02.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.091+0000 7f68a5492700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68a0159050 0x7f68a0156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:02.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.091+0000 7f68a5492700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a01594f0 con 0x7f68a00c4610 2023-06-28T20:08:02.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.091+0000 7f68a5492700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a015cd00 con 0x7f68a014fa30 2023-06-28T20:08:02.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.091+0000 7f68a5492700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a00c5350 con 0x7f68a0159050 2023-06-28T20:08:02.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.091+0000 7f689e7fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a00c4610 0x7f68a0158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:02.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.091+0000 7f689e7fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a00c4610 0x7f68a0158b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54364/0 (socket says 172.21.15.43:54364) 2023-06-28T20:08:02.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.091+0000 7f689e7fc700 1 -- 172.21.15.43:0/2200955313 learned_addr learned my addr 172.21.15.43:0/2200955313 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:02.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.091+0000 7f689effd700 1 --2- 172.21.15.43:0/2200955313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68a014fa30 0x7f68a00c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:02.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.091+0000 7f689f7fe700 1 --2- 172.21.15.43:0/2200955313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68a0159050 0x7f68a0156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:02.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.092+0000 7f689e7fc700 1 -- 172.21.15.43:0/2200955313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68a0159050 msgr2=0x7f68a0156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:02.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.092+0000 7f689e7fc700 1 --2- 172.21.15.43:0/2200955313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68a0159050 0x7f68a0156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.092+0000 7f689e7fc700 1 -- 172.21.15.43:0/2200955313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68a014fa30 msgr2=0x7f68a00c40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:02.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.092+0000 7f689e7fc700 1 --2- 172.21.15.43:0/2200955313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68a014fa30 0x7f68a00c40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.092+0000 7f689e7fc700 1 -- 172.21.15.43:0/2200955313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a00c4ab0 con 0x7f68a00c4610 2023-06-28T20:08:02.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.092+0000 7f689e7fc700 1 --2- 172.21.15.43:0/2200955313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a00c4610 0x7f68a0158b10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6890011800 tx=0x7f6890011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:02.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.092+0000 7f68877fe700 1 -- 172.21.15.43:0/2200955313 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890015870 con 0x7f68a00c4610 2023-06-28T20:08:02.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.092+0000 7f68877fe700 1 -- 172.21.15.43:0/2200955313 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68900159d0 con 0x7f68a00c4610 2023-06-28T20:08:02.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.092+0000 7f68877fe700 1 -- 172.21.15.43:0/2200955313 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890015d80 con 0x7f68a00c4610 2023-06-28T20:08:02.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.092+0000 7f68a5492700 1 -- 172.21.15.43:0/2200955313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a00c4d90 con 0x7f68a00c4610 2023-06-28T20:08:02.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.092+0000 7f68a5492700 1 -- 172.21.15.43:0/2200955313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a014fe00 con 0x7f68a00c4610 2023-06-28T20:08:02.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.093+0000 7f68a5492700 1 -- 172.21.15.43:0/2200955313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68a015cad0 con 0x7f68a00c4610 2023-06-28T20:08:02.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.094+0000 7f68877fe700 1 -- 172.21.15.43:0/2200955313 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6890013400 con 0x7f68a00c4610 2023-06-28T20:08:02.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.094+0000 7f68877fe700 1 --2- 172.21.15.43:0/2200955313 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f688c06ace0 0x7f688c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:02.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.094+0000 7f68877fe700 1 -- 172.21.15.43:0/2200955313 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6890089540 con 0x7f68a00c4610 2023-06-28T20:08:02.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.094+0000 7f689effd700 1 --2- 172.21.15.43:0/2200955313 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f688c06ace0 0x7f688c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:02.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.095+0000 7f689effd700 1 --2- 172.21.15.43:0/2200955313 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f688c06ace0 0x7f688c06d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6894004670 tx=0x7f6894008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:02.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.099+0000 7f68877fe700 1 -- 172.21.15.43:0/2200955313 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f68a015cad0 con 0x7f68a00c4610 2023-06-28T20:08:02.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.297+0000 7f68a5492700 1 -- 172.21.15.43:0/2200955313 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68a015cad0 con 0x7f68a00c4610 2023-06-28T20:08:02.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.298+0000 7f68877fe700 1 -- 172.21.15.43:0/2200955313 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18221 (secure 0 0 0) 0x7f68a015cad0 con 0x7f68a00c4610 2023-06-28T20:08:02.303 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:02.303 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:07:22.377248+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:07:18.624759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-28T20:07:19.760017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:08:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.300+0000 7f68a5492700 1 -- 172.21.15.43:0/2200955313 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f688c06ace0 msgr2=0x7f688c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.301+0000 7f68a5492700 1 --2- 172.21.15.43:0/2200955313 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f688c06ace0 0x7f688c06d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6894004670 tx=0x7f6894008040 comp rx=0 tx=0).stop 2023-06-28T20:08:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.301+0000 7f68a5492700 1 -- 172.21.15.43:0/2200955313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a00c4610 msgr2=0x7f68a0158b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.301+0000 7f68a5492700 1 --2- 172.21.15.43:0/2200955313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a00c4610 0x7f68a0158b10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6890011800 tx=0x7f6890011b10 comp rx=0 tx=0).stop 2023-06-28T20:08:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.301+0000 7f68a5492700 1 -- 172.21.15.43:0/2200955313 shutdown_connections 2023-06-28T20:08:02.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.301+0000 7f68a5492700 1 --2- 172.21.15.43:0/2200955313 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f688c06ace0 0x7f688c06d190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.301+0000 7f68a5492700 1 --2- 172.21.15.43:0/2200955313 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f68a0159050 0x7f68a0156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.301+0000 7f68a5492700 1 --2- 172.21.15.43:0/2200955313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68a014fa30 0x7f68a00c40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.301+0000 7f68a5492700 1 --2- 172.21.15.43:0/2200955313 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68a00c4610 0x7f68a0158b10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.301+0000 7f68a5492700 1 -- 172.21.15.43:0/2200955313 >> 172.21.15.43:0/2200955313 conn(0x7f68a00b1720 msgr2=0x7f68a00b2a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.301+0000 7f68a5492700 1 -- 172.21.15.43:0/2200955313 shutdown_connections 2023-06-28T20:08:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.301+0000 7f68a5492700 1 -- 172.21.15.43:0/2200955313 wait complete. 2023-06-28T20:08:02.317 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-28T20:07:13.580969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-28T20:07:14.038358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-28T20:07:18.624759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-28T20:07:19.760017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-28T20:08:02.318 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-28T20:08:02.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.441+0000 7f43d3839700 1 Processor -- start 2023-06-28T20:08:02.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.442+0000 7f43d3839700 1 -- start start 2023-06-28T20:08:02.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.442+0000 7f43d3839700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43cc06d310 0x7f43cc06cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:02.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.442+0000 7f43d3839700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f43cc15cae0 0x7f43cc06d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:02.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.442+0000 7f43d3839700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43cc0d0680 con 0x7f43cc0cfe00 2023-06-28T20:08:02.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.442+0000 7f43d3839700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43cc06dc20 con 0x7f43cc15cae0 2023-06-28T20:08:02.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.442+0000 7f43d3839700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc06bb20 con 0x7f43cc06d310 2023-06-28T20:08:02.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.443+0000 7f43d0dd4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f43cc15cae0 0x7f43cc06d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:02.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.443+0000 7f43d0dd4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f43cc15cae0 0x7f43cc06d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54370/0 (socket says 172.21.15.43:54370) 2023-06-28T20:08:02.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.443+0000 7f43d0dd4700 1 -- 172.21.15.43:0/2701188441 learned_addr learned my addr 172.21.15.43:0/2701188441 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:02.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.443+0000 7f43d15d5700 1 --2- 172.21.15.43:0/2701188441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43cc06d310 0x7f43cc06cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:02.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.443+0000 7f43d0dd4700 1 -- 172.21.15.43:0/2701188441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43cc06d310 msgr2=0x7f43cc06cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:02.448 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.443+0000 7f43d0dd4700 1 --2- 172.21.15.43:0/2701188441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43cc06d310 0x7f43cc06cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.443+0000 7f43d0dd4700 1 -- 172.21.15.43:0/2701188441 >> v1:172.21.15.178:6789/0 conn(0x7f43cc0cfe00 legacy=0x7f43cc0ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:02.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.443+0000 7f43d0dd4700 1 -- 172.21.15.43:0/2701188441 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc06be00 con 0x7f43cc15cae0 2023-06-28T20:08:02.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.443+0000 7f43d0dd4700 1 --2- 172.21.15.43:0/2701188441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43cc15cae0 0x7f43cc06d6e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f43c80098f0 tx=0x7f43c8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c8bcb6cc989e69 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:02.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.443+0000 7f43c3fff700 1 -- 172.21.15.43:0/2701188441 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c800b8a0 con 0x7f43cc15cae0 2023-06-28T20:08:02.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.443+0000 7f43c3fff700 1 -- 172.21.15.43:0/2701188441 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c800ba00 con 0x7f43cc15cae0 2023-06-28T20:08:02.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.443+0000 7f43c3fff700 1 -- 172.21.15.43:0/2701188441 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c800bd90 con 0x7f43cc15cae0 2023-06-28T20:08:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.444+0000 7f43d3839700 1 -- 172.21.15.43:0/2701188441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43cc15cae0 msgr2=0x7f43cc06d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.444+0000 7f43d3839700 1 --2- 172.21.15.43:0/2701188441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43cc15cae0 0x7f43cc06d6e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f43c80098f0 tx=0x7f43c8009c00 comp rx=0 tx=0).stop 2023-06-28T20:08:02.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.444+0000 7f43d3839700 1 -- 172.21.15.43:0/2701188441 shutdown_connections 2023-06-28T20:08:02.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.444+0000 7f43d3839700 1 --2- 172.21.15.43:0/2701188441 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43cc06d310 0x7f43cc06cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.444+0000 7f43d3839700 1 --2- 172.21.15.43:0/2701188441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43cc15cae0 0x7f43cc06d6e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.444+0000 7f43d3839700 1 -- 172.21.15.43:0/2701188441 >> 172.21.15.43:0/2701188441 conn(0x7f43cc0664b0 msgr2=0x7f43cc0d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:02.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.444+0000 7f43d3839700 1 -- 172.21.15.43:0/2701188441 shutdown_connections 2023-06-28T20:08:02.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.444+0000 7f43d3839700 1 -- 172.21.15.43:0/2701188441 wait complete. 2023-06-28T20:08:02.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.445+0000 7f43d3839700 1 Processor -- start 2023-06-28T20:08:02.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.445+0000 7f43d3839700 1 -- start start 2023-06-28T20:08:02.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.446+0000 7f43d3839700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43cc06d310 0x7f43cc15a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:02.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.446+0000 7f43d3839700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43cc0cfe00 0x7f43cc0cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:02.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.446+0000 7f43d3839700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43cc15cae0 0x7f43cc0b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:02.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.446+0000 7f43d3839700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc0cc830 con 0x7f43cc15cae0 2023-06-28T20:08:02.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.446+0000 7f43d3839700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc0cd710 con 0x7f43cc06d310 2023-06-28T20:08:02.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.446+0000 7f43d3839700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc15aa00 con 0x7f43cc0cfe00 2023-06-28T20:08:02.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.446+0000 7f43d1dd6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43cc15cae0 0x7f43cc0b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:02.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.446+0000 7f43d1dd6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43cc15cae0 0x7f43cc0b0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54380/0 (socket says 172.21.15.43:54380) 2023-06-28T20:08:02.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.446+0000 7f43d1dd6700 1 -- 172.21.15.43:0/1810226803 learned_addr learned my addr 172.21.15.43:0/1810226803 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:02.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.446+0000 7f43d0dd4700 1 --2- 172.21.15.43:0/1810226803 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43cc0cfe00 0x7f43cc0cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:02.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.446+0000 7f43d15d5700 1 --2- 172.21.15.43:0/1810226803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43cc06d310 0x7f43cc15a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:02.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.446+0000 7f43d1dd6700 1 -- 172.21.15.43:0/1810226803 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43cc0cfe00 msgr2=0x7f43cc0cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.446+0000 7f43d1dd6700 1 --2- 172.21.15.43:0/1810226803 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43cc0cfe00 0x7f43cc0cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.446+0000 7f43d1dd6700 1 -- 172.21.15.43:0/1810226803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43cc06d310 msgr2=0x7f43cc15a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.447+0000 7f43d1dd6700 1 --2- 172.21.15.43:0/1810226803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43cc06d310 0x7f43cc15a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.447+0000 7f43d1dd6700 1 -- 172.21.15.43:0/1810226803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c8009550 con 0x7f43cc15cae0 2023-06-28T20:08:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.447+0000 7f43d1dd6700 1 --2- 172.21.15.43:0/1810226803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43cc15cae0 0x7f43cc0b0660 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f43c400cc80 tx=0x7f43c400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.447+0000 7f43c1ffb700 1 -- 172.21.15.43:0/1810226803 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4003c00 con 0x7f43cc15cae0 2023-06-28T20:08:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.447+0000 7f43c1ffb700 1 -- 172.21.15.43:0/1810226803 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c4003d60 con 0x7f43cc15cae0 2023-06-28T20:08:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.447+0000 7f43d3839700 1 -- 172.21.15.43:0/1810226803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43cc0ccb00 con 0x7f43cc15cae0 2023-06-28T20:08:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.447+0000 7f43c1ffb700 1 -- 172.21.15.43:0/1810226803 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c400aaa0 con 0x7f43cc15cae0 2023-06-28T20:08:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.447+0000 7f43d3839700 1 -- 172.21.15.43:0/1810226803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43cc0cd0c0 con 0x7f43cc15cae0 2023-06-28T20:08:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.448+0000 7f43d3839700 1 -- 172.21.15.43:0/1810226803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43cc0cd8a0 con 0x7f43cc15cae0 2023-06-28T20:08:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.448+0000 7f43c1ffb700 1 -- 172.21.15.43:0/1810226803 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f43c400ac00 con 0x7f43cc15cae0 2023-06-28T20:08:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.449+0000 7f43c1ffb700 1 --2- 172.21.15.43:0/1810226803 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f43b006ace0 0x7f43b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.449+0000 7f43c1ffb700 1 -- 172.21.15.43:0/1810226803 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f43c4089340 con 0x7f43cc15cae0 2023-06-28T20:08:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.449+0000 7f43d15d5700 1 --2- 172.21.15.43:0/1810226803 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f43b006ace0 0x7f43b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.449+0000 7f43d15d5700 1 --2- 172.21.15.43:0/1810226803 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f43b006ace0 0x7f43b006d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f43bc006fd0 tx=0x7f43bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.453+0000 7f43c1ffb700 1 -- 172.21.15.43:0/1810226803 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f43c4054480 con 0x7f43cc15cae0 2023-06-28T20:08:02.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.651+0000 7f43d3839700 1 -- 172.21.15.43:0/1810226803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f43cc0728b0 con 0x7f43cc15cae0 2023-06-28T20:08:02.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.651+0000 7f43c1ffb700 1 -- 172.21.15.43:0/1810226803 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7f43c4057aa0 con 0x7f43cc15cae0 2023-06-28T20:08:02.655 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-06-28T20:08:02.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.653+0000 7f43d3839700 1 -- 172.21.15.43:0/1810226803 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f43b006ace0 msgr2=0x7f43b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:02.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.653+0000 7f43d3839700 1 --2- 172.21.15.43:0/1810226803 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f43b006ace0 0x7f43b006d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f43bc006fd0 tx=0x7f43bc008040 comp rx=0 tx=0).stop 2023-06-28T20:08:02.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.653+0000 7f43d3839700 1 -- 172.21.15.43:0/1810226803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43cc15cae0 msgr2=0x7f43cc0b0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:02.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.653+0000 7f43d3839700 1 --2- 172.21.15.43:0/1810226803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43cc15cae0 0x7f43cc0b0660 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f43c400cc80 tx=0x7f43c400cf90 comp rx=0 tx=0).stop 2023-06-28T20:08:02.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.653+0000 7f43d3839700 1 -- 172.21.15.43:0/1810226803 shutdown_connections 2023-06-28T20:08:02.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.653+0000 7f43d3839700 1 --2- 172.21.15.43:0/1810226803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43cc15cae0 0x7f43cc0b0660 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.653+0000 7f43d3839700 1 --2- 172.21.15.43:0/1810226803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43cc06d310 0x7f43cc15a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.653+0000 7f43d3839700 1 --2- 172.21.15.43:0/1810226803 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f43b006ace0 0x7f43b006d190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.653+0000 7f43d3839700 1 --2- 172.21.15.43:0/1810226803 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f43cc0cfe00 0x7f43cc0cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.653+0000 7f43d3839700 1 -- 172.21.15.43:0/1810226803 >> 172.21.15.43:0/1810226803 conn(0x7f43cc0664b0 msgr2=0x7f43cc0c7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:02.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.654+0000 7f43d3839700 1 -- 172.21.15.43:0/1810226803 shutdown_connections 2023-06-28T20:08:02.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.654+0000 7f43d3839700 1 -- 172.21.15.43:0/1810226803 wait complete. 2023-06-28T20:08:02.669 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-28T20:08:02.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.793+0000 7fb849e41700 1 Processor -- start 2023-06-28T20:08:02.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.794+0000 7fb849e41700 1 -- start start 2023-06-28T20:08:02.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.794+0000 7fb849e41700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8440aec90 0x7fb844160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:02.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.794+0000 7fb849e41700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb8440af060 0x7fb844070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:02.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.794+0000 7fb849e41700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb844160250 con 0x7fb84406ccc0 2023-06-28T20:08:02.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.794+0000 7fb849e41700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb84415cb30 con 0x7fb8440af060 2023-06-28T20:08:02.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.794+0000 7fb849e41700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb844071410 con 0x7fb8440aec90 2023-06-28T20:08:02.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.794+0000 7fb842ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb8440af060 0x7fb844070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:02.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.795+0000 7fb842ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb8440af060 0x7fb844070bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54386/0 (socket says 172.21.15.43:54386) 2023-06-28T20:08:02.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.795+0000 7fb842ffd700 1 -- 172.21.15.43:0/2820866511 learned_addr learned my addr 172.21.15.43:0/2820866511 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:02.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.795+0000 7fb8437fe700 1 --2- 172.21.15.43:0/2820866511 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8440aec90 0x7fb844160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:02.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.795+0000 7fb842ffd700 1 -- 172.21.15.43:0/2820866511 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8440aec90 msgr2=0x7fb844160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:02.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.795+0000 7fb842ffd700 1 --2- 172.21.15.43:0/2820866511 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8440aec90 0x7fb844160cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.795+0000 7fb842ffd700 1 -- 172.21.15.43:0/2820866511 >> v1:172.21.15.178:6789/0 conn(0x7fb84406ccc0 legacy=0x7fb8440b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:02.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.795+0000 7fb842ffd700 1 -- 172.21.15.43:0/2820866511 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8441604f0 con 0x7fb8440af060 2023-06-28T20:08:02.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.795+0000 7fb8437fe700 1 --2- 172.21.15.43:0/2820866511 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8440aec90 0x7fb844160cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:02.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.795+0000 7fb842ffd700 1 --2- 172.21.15.43:0/2820866511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8440af060 0x7fb844070bc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0098f0 tx=0x7fb82c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace51ae136fda693 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:02.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.795+0000 7fb8427fc700 1 -- 172.21.15.43:0/2820866511 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c00b8a0 con 0x7fb8440af060 2023-06-28T20:08:02.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.795+0000 7fb8427fc700 1 -- 172.21.15.43:0/2820866511 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82c00ba00 con 0x7fb8440af060 2023-06-28T20:08:02.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.795+0000 7fb8427fc700 1 -- 172.21.15.43:0/2820866511 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c00bd90 con 0x7fb8440af060 2023-06-28T20:08:02.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.796+0000 7fb849e41700 1 -- 172.21.15.43:0/2820866511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8440af060 msgr2=0x7fb844070bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:02.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.796+0000 7fb849e41700 1 --2- 172.21.15.43:0/2820866511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8440af060 0x7fb844070bc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0098f0 tx=0x7fb82c009c00 comp rx=0 tx=0).stop 2023-06-28T20:08:02.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.796+0000 7fb849e41700 1 -- 172.21.15.43:0/2820866511 shutdown_connections 2023-06-28T20:08:02.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.796+0000 7fb849e41700 1 --2- 172.21.15.43:0/2820866511 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb8440aec90 0x7fb844160cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.796+0000 7fb849e41700 1 --2- 172.21.15.43:0/2820866511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8440af060 0x7fb844070bc0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.796+0000 7fb849e41700 1 -- 172.21.15.43:0/2820866511 >> 172.21.15.43:0/2820866511 conn(0x7fb84406b000 msgr2=0x7fb84406b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:02.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.796+0000 7fb849e41700 1 -- 172.21.15.43:0/2820866511 shutdown_connections 2023-06-28T20:08:02.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.796+0000 7fb849e41700 1 -- 172.21.15.43:0/2820866511 wait complete. 2023-06-28T20:08:02.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.797+0000 7fb849e41700 1 Processor -- start 2023-06-28T20:08:02.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.797+0000 7fb849e41700 1 -- start start 2023-06-28T20:08:02.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.798+0000 7fb849e41700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb84406ccc0 0x7fb844150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:02.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.798+0000 7fb849e41700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8440aec90 0x7fb8440b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:02.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.798+0000 7fb849e41700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8440af060 0x7fb8440cef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:02.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.798+0000 7fb849e41700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8440c72b0 con 0x7fb8440af060 2023-06-28T20:08:02.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.798+0000 7fb849e41700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8440c6c60 con 0x7fb8440aec90 2023-06-28T20:08:02.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.798+0000 7fb849e41700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8440b4920 con 0x7fb84406ccc0 2023-06-28T20:08:02.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.798+0000 7fb843fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8440af060 0x7fb8440cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:02.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.798+0000 7fb843fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8440af060 0x7fb8440cef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54394/0 (socket says 172.21.15.43:54394) 2023-06-28T20:08:02.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.798+0000 7fb843fff700 1 -- 172.21.15.43:0/1597012475 learned_addr learned my addr 172.21.15.43:0/1597012475 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:02.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.798+0000 7fb842ffd700 1 --2- 172.21.15.43:0/1597012475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8440aec90 0x7fb8440b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:02.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.798+0000 7fb8437fe700 1 --2- 172.21.15.43:0/1597012475 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb84406ccc0 0x7fb844150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:02.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.798+0000 7fb843fff700 1 -- 172.21.15.43:0/1597012475 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb84406ccc0 msgr2=0x7fb844150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:02.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.798+0000 7fb843fff700 1 --2- 172.21.15.43:0/1597012475 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb84406ccc0 0x7fb844150d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.798+0000 7fb843fff700 1 -- 172.21.15.43:0/1597012475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8440aec90 msgr2=0x7fb8440b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:02.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.798+0000 7fb843fff700 1 --2- 172.21.15.43:0/1597012475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8440aec90 0x7fb8440b0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:02.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.798+0000 7fb843fff700 1 -- 172.21.15.43:0/1597012475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82c009550 con 0x7fb8440af060 2023-06-28T20:08:02.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.799+0000 7fb843fff700 1 --2- 172.21.15.43:0/1597012475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8440af060 0x7fb8440cef80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb83c00c500 tx=0x7fb83c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:02.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.799+0000 7fb83bfff700 1 -- 172.21.15.43:0/1597012475 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c0052d0 con 0x7fb8440af060 2023-06-28T20:08:02.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.799+0000 7fb83bfff700 1 -- 172.21.15.43:0/1597012475 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83c005430 con 0x7fb8440af060 2023-06-28T20:08:02.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.799+0000 7fb849e41700 1 -- 172.21.15.43:0/1597012475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8440c6ed0 con 0x7fb8440af060 2023-06-28T20:08:02.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.799+0000 7fb83bfff700 1 -- 172.21.15.43:0/1597012475 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c0038c0 con 0x7fb8440af060 2023-06-28T20:08:02.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.799+0000 7fb849e41700 1 -- 172.21.15.43:0/1597012475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8440b1340 con 0x7fb8440af060 2023-06-28T20:08:02.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.800+0000 7fb849e41700 1 -- 172.21.15.43:0/1597012475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8440c73f0 con 0x7fb8440af060 2023-06-28T20:08:02.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.800+0000 7fb83bfff700 1 -- 172.21.15.43:0/1597012475 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb83c003a20 con 0x7fb8440af060 2023-06-28T20:08:02.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.801+0000 7fb83bfff700 1 --2- 172.21.15.43:0/1597012475 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb82406ace0 0x7fb82406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:02.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.801+0000 7fb83bfff700 1 -- 172.21.15.43:0/1597012475 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb83c088730 con 0x7fb8440af060 2023-06-28T20:08:02.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.801+0000 7fb8437fe700 1 --2- 172.21.15.43:0/1597012475 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb82406ace0 0x7fb82406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:02.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.801+0000 7fb8437fe700 1 --2- 172.21.15.43:0/1597012475 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb82406ace0 0x7fb82406d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb834003c10 tx=0x7fb834009890 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:02.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:02.805+0000 7fb83bfff700 1 -- 172.21.15.43:0/1597012475 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb8440c73f0 con 0x7fb8440af060 2023-06-28T20:08:03.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.002+0000 7fb849e41700 1 -- 172.21.15.43:0/1597012475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb8440c73f0 con 0x7fb8440af060 2023-06-28T20:08:03.007 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.003+0000 7fb83bfff700 1 -- 172.21.15.43:0/1597012475 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb8440c73f0 con 0x7fb8440af060 2023-06-28T20:08:03.007 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-06-28T20:08:03.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.005+0000 7fb849e41700 1 -- 172.21.15.43:0/1597012475 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb82406ace0 msgr2=0x7fb82406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:03.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.005+0000 7fb849e41700 1 --2- 172.21.15.43:0/1597012475 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb82406ace0 0x7fb82406d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb834003c10 tx=0x7fb834009890 comp rx=0 tx=0).stop 2023-06-28T20:08:03.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.005+0000 7fb849e41700 1 -- 172.21.15.43:0/1597012475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8440af060 msgr2=0x7fb8440cef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:03.010 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.005+0000 7fb849e41700 1 --2- 172.21.15.43:0/1597012475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8440af060 0x7fb8440cef80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb83c00c500 tx=0x7fb83c00c810 comp rx=0 tx=0).stop 2023-06-28T20:08:03.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.005+0000 7fb849e41700 1 -- 172.21.15.43:0/1597012475 shutdown_connections 2023-06-28T20:08:03.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.005+0000 7fb849e41700 1 --2- 172.21.15.43:0/1597012475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8440af060 0x7fb8440cef80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.005+0000 7fb849e41700 1 --2- 172.21.15.43:0/1597012475 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb82406ace0 0x7fb82406d190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.005+0000 7fb849e41700 1 --2- 172.21.15.43:0/1597012475 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb84406ccc0 0x7fb844150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.005+0000 7fb849e41700 1 --2- 172.21.15.43:0/1597012475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8440aec90 0x7fb8440b0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.005+0000 7fb849e41700 1 -- 172.21.15.43:0/1597012475 >> 172.21.15.43:0/1597012475 conn(0x7fb84406b000 msgr2=0x7fb84415bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:03.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.005+0000 7fb849e41700 1 -- 172.21.15.43:0/1597012475 shutdown_connections 2023-06-28T20:08:03.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.005+0000 7fb849e41700 1 -- 172.21.15.43:0/1597012475 wait complete. 2023-06-28T20:08:03.021 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-28T20:08:03.150 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.146+0000 7f106db8c700 1 Processor -- start 2023-06-28T20:08:03.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.147+0000 7f106db8c700 1 -- start start 2023-06-28T20:08:03.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.147+0000 7f106db8c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10680cf6e0 0x7f10680ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:03.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.147+0000 7f106db8c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f10680cca10 0x7f1068071270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:03.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.147+0000 7f106db8c700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f10680b1920 0x7f10680b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:03.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.147+0000 7f106db8c700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10681609c0 con 0x7f10680cca10 2023-06-28T20:08:03.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.147+0000 7f106db8c700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10680b22c0 con 0x7f10680b1920 2023-06-28T20:08:03.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.147+0000 7f106db8c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10680717b0 con 0x7f10680cf6e0 2023-06-28T20:08:03.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.147+0000 7f1066ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f10680cca10 0x7f1068071270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:03.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.148+0000 7f1066ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f10680cca10 0x7f1068071270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54400/0 (socket says 172.21.15.43:54400) 2023-06-28T20:08:03.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.148+0000 7f1066ffd700 1 -- 172.21.15.43:0/2681811815 learned_addr learned my addr 172.21.15.43:0/2681811815 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:03.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.148+0000 7f10677fe700 1 --2- 172.21.15.43:0/2681811815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10680cf6e0 0x7f10680ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:03.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.148+0000 7f1067fff700 1 --2- 172.21.15.43:0/2681811815 >> v2:172.21.15.178:3300/0 conn(0x7f10680b1920 0x7f10680b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:03.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.148+0000 7f1066ffd700 1 -- 172.21.15.43:0/2681811815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10680cf6e0 msgr2=0x7f10680ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:03.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.148+0000 7f1066ffd700 1 --2- 172.21.15.43:0/2681811815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10680cf6e0 0x7f10680ce590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.148+0000 7f1066ffd700 1 -- 172.21.15.43:0/2681811815 >> v2:172.21.15.178:3300/0 conn(0x7f10680b1920 msgr2=0x7f10680b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:03.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.148+0000 7f1066ffd700 1 --2- 172.21.15.43:0/2681811815 >> v2:172.21.15.178:3300/0 conn(0x7f10680b1920 0x7f10680b1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.148+0000 7f1066ffd700 1 -- 172.21.15.43:0/2681811815 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1068160e90 con 0x7f10680cca10 2023-06-28T20:08:03.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.148+0000 7f1067fff700 1 --2- 172.21.15.43:0/2681811815 >> v2:172.21.15.178:3300/0 conn(0x7f10680b1920 0x7f10680b1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:03.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.148+0000 7f1066ffd700 1 --2- 172.21.15.43:0/2681811815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10680cca10 0x7f1068071270 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f10680b15d0 tx=0x7f105c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795a40bd43397400 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:03.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.148+0000 7f10667fc700 1 -- 172.21.15.43:0/2681811815 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c00b870 con 0x7f10680cca10 2023-06-28T20:08:03.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.148+0000 7f10667fc700 1 -- 172.21.15.43:0/2681811815 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105c00b9d0 con 0x7f10680cca10 2023-06-28T20:08:03.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.148+0000 7f10667fc700 1 -- 172.21.15.43:0/2681811815 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105c00bd60 con 0x7f10680cca10 2023-06-28T20:08:03.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.149+0000 7f106db8c700 1 -- 172.21.15.43:0/2681811815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10680cca10 msgr2=0x7f1068071270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:03.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.149+0000 7f106db8c700 1 --2- 172.21.15.43:0/2681811815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10680cca10 0x7f1068071270 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f10680b15d0 tx=0x7f105c009bd0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.149+0000 7f106db8c700 1 -- 172.21.15.43:0/2681811815 shutdown_connections 2023-06-28T20:08:03.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.149+0000 7f106db8c700 1 --2- 172.21.15.43:0/2681811815 >> v2:172.21.15.178:3300/0 conn(0x7f10680b1920 0x7f10680b1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.149+0000 7f106db8c700 1 --2- 172.21.15.43:0/2681811815 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10680cf6e0 0x7f10680ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.149+0000 7f106db8c700 1 --2- 172.21.15.43:0/2681811815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10680cca10 0x7f1068071270 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.149+0000 7f106db8c700 1 -- 172.21.15.43:0/2681811815 >> 172.21.15.43:0/2681811815 conn(0x7f106806b000 msgr2=0x7f106806b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:03.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.149+0000 7f106db8c700 1 -- 172.21.15.43:0/2681811815 shutdown_connections 2023-06-28T20:08:03.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.149+0000 7f106db8c700 1 -- 172.21.15.43:0/2681811815 wait complete. 2023-06-28T20:08:03.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.150+0000 7f106db8c700 1 Processor -- start 2023-06-28T20:08:03.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.150+0000 7f106db8c700 1 -- start start 2023-06-28T20:08:03.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.151+0000 7f106db8c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10680b1920 0x7f10680cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:03.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.151+0000 7f106db8c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10680cca10 0x7f106815a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:03.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.151+0000 7f106db8c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10680cf6e0 0x7f106815fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:03.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.151+0000 7f106db8c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10680b2940 con 0x7f10680cf6e0 2023-06-28T20:08:03.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.151+0000 7f106db8c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10681f55f0 con 0x7f10680b1920 2023-06-28T20:08:03.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.151+0000 7f106db8c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10681f58d0 con 0x7f10680cca10 2023-06-28T20:08:03.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.151+0000 7f1067fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10680cf6e0 0x7f106815fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:03.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.151+0000 7f1067fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10680cf6e0 0x7f106815fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54408/0 (socket says 172.21.15.43:54408) 2023-06-28T20:08:03.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.151+0000 7f1067fff700 1 -- 172.21.15.43:0/2111982783 learned_addr learned my addr 172.21.15.43:0/2111982783 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:03.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.151+0000 7f10677fe700 1 --2- 172.21.15.43:0/2111982783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10680b1920 0x7f10680cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:03.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.151+0000 7f1066ffd700 1 --2- 172.21.15.43:0/2111982783 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10680cca10 0x7f106815a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:03.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.151+0000 7f1067fff700 1 -- 172.21.15.43:0/2111982783 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10680cca10 msgr2=0x7f106815a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:03.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.151+0000 7f1067fff700 1 --2- 172.21.15.43:0/2111982783 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10680cca10 0x7f106815a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.151+0000 7f1067fff700 1 -- 172.21.15.43:0/2111982783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10680b1920 msgr2=0x7f10680cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:03.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.151+0000 7f1067fff700 1 --2- 172.21.15.43:0/2111982783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10680b1920 0x7f10680cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.151+0000 7f1067fff700 1 -- 172.21.15.43:0/2111982783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c009550 con 0x7f10680cf6e0 2023-06-28T20:08:03.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.151+0000 7f10677fe700 1 --2- 172.21.15.43:0/2111982783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10680b1920 0x7f10680cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:03.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.151+0000 7f1067fff700 1 --2- 172.21.15.43:0/2111982783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10680cf6e0 0x7f106815fd90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f106000cef0 tx=0x7f106000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:03.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.152+0000 7f104ffff700 1 -- 172.21.15.43:0/2111982783 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106000add0 con 0x7f10680cf6e0 2023-06-28T20:08:03.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.152+0000 7f104ffff700 1 -- 172.21.15.43:0/2111982783 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1060015440 con 0x7f10680cf6e0 2023-06-28T20:08:03.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.152+0000 7f106db8c700 1 -- 172.21.15.43:0/2111982783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10681f5bb0 con 0x7f10680cf6e0 2023-06-28T20:08:03.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.152+0000 7f104ffff700 1 -- 172.21.15.43:0/2111982783 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10600157f0 con 0x7f10680cf6e0 2023-06-28T20:08:03.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.152+0000 7f106db8c700 1 -- 172.21.15.43:0/2111982783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10680cc070 con 0x7f10680cf6e0 2023-06-28T20:08:03.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.153+0000 7f106db8c700 1 -- 172.21.15.43:0/2111982783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10680728b0 con 0x7f10680cf6e0 2023-06-28T20:08:03.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.153+0000 7f104ffff700 1 -- 172.21.15.43:0/2111982783 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f106000d430 con 0x7f10680cf6e0 2023-06-28T20:08:03.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.153+0000 7f104ffff700 1 --2- 172.21.15.43:0/2111982783 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f105406ace0 0x7f105406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:03.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.154+0000 7f104ffff700 1 -- 172.21.15.43:0/2111982783 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f10600899a0 con 0x7f10680cf6e0 2023-06-28T20:08:03.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.154+0000 7f10677fe700 1 --2- 172.21.15.43:0/2111982783 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f105406ace0 0x7f105406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:03.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.154+0000 7f10677fe700 1 --2- 172.21.15.43:0/2111982783 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f105406ace0 0x7f105406d190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f105800a9b0 tx=0x7f1058008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:03.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.158+0000 7f104ffff700 1 -- 172.21.15.43:0/2111982783 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1060054ae0 con 0x7f10680cf6e0 2023-06-28T20:08:03.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.355+0000 7f106db8c700 1 -- 172.21.15.43:0/2111982783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f10680cc640 con 0x7f10680cf6e0 2023-06-28T20:08:03.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.355+0000 7f104ffff700 1 -- 172.21.15.43:0/2111982783 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f106001a020 con 0x7f10680cf6e0 2023-06-28T20:08:03.360 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-06-28T20:08:03.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.357+0000 7f106db8c700 1 -- 172.21.15.43:0/2111982783 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f105406ace0 msgr2=0x7f105406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:03.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.357+0000 7f106db8c700 1 --2- 172.21.15.43:0/2111982783 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f105406ace0 0x7f105406d190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f105800a9b0 tx=0x7f1058008040 comp rx=0 tx=0).stop 2023-06-28T20:08:03.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.357+0000 7f106db8c700 1 -- 172.21.15.43:0/2111982783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10680cf6e0 msgr2=0x7f106815fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:03.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.357+0000 7f106db8c700 1 --2- 172.21.15.43:0/2111982783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10680cf6e0 0x7f106815fd90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f106000cef0 tx=0x7f106000a340 comp rx=0 tx=0).stop 2023-06-28T20:08:03.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.358+0000 7f106db8c700 1 -- 172.21.15.43:0/2111982783 shutdown_connections 2023-06-28T20:08:03.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.358+0000 7f106db8c700 1 --2- 172.21.15.43:0/2111982783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10680cf6e0 0x7f106815fd90 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.358+0000 7f106db8c700 1 --2- 172.21.15.43:0/2111982783 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f10680b1920 0x7f10680cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.358+0000 7f106db8c700 1 --2- 172.21.15.43:0/2111982783 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f105406ace0 0x7f105406d190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.358+0000 7f106db8c700 1 --2- 172.21.15.43:0/2111982783 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f10680cca10 0x7f106815a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.358+0000 7f106db8c700 1 -- 172.21.15.43:0/2111982783 >> 172.21.15.43:0/2111982783 conn(0x7f106806b000 msgr2=0x7f106815e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:03.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.358+0000 7f106db8c700 1 -- 172.21.15.43:0/2111982783 shutdown_connections 2023-06-28T20:08:03.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.358+0000 7f106db8c700 1 -- 172.21.15.43:0/2111982783 wait complete. 2023-06-28T20:08:03.374 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-28T20:08:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.500+0000 7f36cbf57700 1 Processor -- start 2023-06-28T20:08:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.500+0000 7f36cbf57700 1 -- start start 2023-06-28T20:08:03.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.501+0000 7f36cbf57700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f36c415cae0 0x7f36c40cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:03.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.501+0000 7f36cbf57700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c40d0680 con 0x7f36c406d310 2023-06-28T20:08:03.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.501+0000 7f36cbf57700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c406bb20 con 0x7f36c40cfe00 2023-06-28T20:08:03.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.501+0000 7f36cbf57700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36c406be00 con 0x7f36c415cae0 2023-06-28T20:08:03.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.501+0000 7f36c94f2700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f36c415cae0 0x7f36c40cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:03.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.501+0000 7f36c9cf3700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f36c406d310 0x7f36c406d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43598/0 (socket says 172.21.15.43:43598) 2023-06-28T20:08:03.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.501+0000 7f36c9cf3700 1 -- 172.21.15.43:0/988806888 learned_addr learned my addr 172.21.15.43:0/988806888 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:03.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.501+0000 7f36c94f2700 1 -- 172.21.15.43:0/988806888 >> v1:172.21.15.178:6789/0 conn(0x7f36c40cfe00 legacy=0x7f36c40ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:03.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.501+0000 7f36c94f2700 1 -- 172.21.15.43:0/988806888 >> v1:172.21.15.43:6789/0 conn(0x7f36c406d310 legacy=0x7f36c406d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:03.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.501+0000 7f36c94f2700 1 -- 172.21.15.43:0/988806888 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c406c0e0 con 0x7f36c415cae0 2023-06-28T20:08:03.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.501+0000 7f36c94f2700 1 --2- 172.21.15.43:0/988806888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36c415cae0 0x7f36c40cf7e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f36c0009840 tx=0x7f36c0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3e62f254ac097b server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:03.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.502+0000 7f36c8cf1700 1 -- 172.21.15.43:0/988806888 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c0004230 con 0x7f36c415cae0 2023-06-28T20:08:03.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.502+0000 7f36c8cf1700 1 -- 172.21.15.43:0/988806888 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c0004390 con 0x7f36c415cae0 2023-06-28T20:08:03.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.502+0000 7f36c8cf1700 1 -- 172.21.15.43:0/988806888 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c00036a0 con 0x7f36c415cae0 2023-06-28T20:08:03.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.502+0000 7f36cbf57700 1 -- 172.21.15.43:0/988806888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36c415cae0 msgr2=0x7f36c40cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:03.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.502+0000 7f36cbf57700 1 --2- 172.21.15.43:0/988806888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36c415cae0 0x7f36c40cf7e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f36c0009840 tx=0x7f36c0009b50 comp rx=0 tx=0).stop 2023-06-28T20:08:03.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.502+0000 7f36cbf57700 1 -- 172.21.15.43:0/988806888 shutdown_connections 2023-06-28T20:08:03.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.502+0000 7f36cbf57700 1 --2- 172.21.15.43:0/988806888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36c415cae0 0x7f36c40cf7e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.502+0000 7f36cbf57700 1 -- 172.21.15.43:0/988806888 >> 172.21.15.43:0/988806888 conn(0x7f36c40664b0 msgr2=0x7f36c40d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.502+0000 7f36cbf57700 1 -- 172.21.15.43:0/988806888 shutdown_connections 2023-06-28T20:08:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.502+0000 7f36cbf57700 1 -- 172.21.15.43:0/988806888 wait complete. 2023-06-28T20:08:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.503+0000 7f36cbf57700 1 Processor -- start 2023-06-28T20:08:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.504+0000 7f36cbf57700 1 -- start start 2023-06-28T20:08:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.504+0000 7f36cbf57700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c406d310 0x7f36c414c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.504+0000 7f36cbf57700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c40cfe00 0x7f36c41f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:03.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.504+0000 7f36cbf57700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36c415cae0 0x7f36c41488c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:03.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.504+0000 7f36cbf57700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c414a1d0 con 0x7f36c415cae0 2023-06-28T20:08:03.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.504+0000 7f36cbf57700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c41f2180 con 0x7f36c406d310 2023-06-28T20:08:03.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.504+0000 7f36cbf57700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c4149e30 con 0x7f36c40cfe00 2023-06-28T20:08:03.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.504+0000 7f36ca4f4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36c415cae0 0x7f36c41488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:03.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.504+0000 7f36ca4f4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36c415cae0 0x7f36c41488c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54422/0 (socket says 172.21.15.43:54422) 2023-06-28T20:08:03.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.504+0000 7f36ca4f4700 1 -- 172.21.15.43:0/3044877392 learned_addr learned my addr 172.21.15.43:0/3044877392 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:03.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.505+0000 7f36c9cf3700 1 --2- 172.21.15.43:0/3044877392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c406d310 0x7f36c414c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:03.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.505+0000 7f36c94f2700 1 --2- 172.21.15.43:0/3044877392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c40cfe00 0x7f36c41f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:03.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.505+0000 7f36ca4f4700 1 -- 172.21.15.43:0/3044877392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c40cfe00 msgr2=0x7f36c41f1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:03.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.505+0000 7f36ca4f4700 1 --2- 172.21.15.43:0/3044877392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c40cfe00 0x7f36c41f1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.505+0000 7f36ca4f4700 1 -- 172.21.15.43:0/3044877392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c406d310 msgr2=0x7f36c414c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:03.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.505+0000 7f36ca4f4700 1 --2- 172.21.15.43:0/3044877392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c406d310 0x7f36c414c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.505+0000 7f36ca4f4700 1 -- 172.21.15.43:0/3044877392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c00094a0 con 0x7f36c415cae0 2023-06-28T20:08:03.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.505+0000 7f36ca4f4700 1 --2- 172.21.15.43:0/3044877392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36c415cae0 0x7f36c41488c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00cc10 tx=0x7f36bc00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:03.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.505+0000 7f36ba7fc700 1 -- 172.21.15.43:0/3044877392 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc00aa60 con 0x7f36c415cae0 2023-06-28T20:08:03.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.505+0000 7f36ba7fc700 1 -- 172.21.15.43:0/3044877392 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36bc00abc0 con 0x7f36c415cae0 2023-06-28T20:08:03.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.505+0000 7f36cbf57700 1 -- 172.21.15.43:0/3044877392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c4149220 con 0x7f36c415cae0 2023-06-28T20:08:03.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.505+0000 7f36ba7fc700 1 -- 172.21.15.43:0/3044877392 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc012420 con 0x7f36c415cae0 2023-06-28T20:08:03.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.505+0000 7f36cbf57700 1 -- 172.21.15.43:0/3044877392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c41497e0 con 0x7f36c415cae0 2023-06-28T20:08:03.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.506+0000 7f36cbf57700 1 -- 172.21.15.43:0/3044877392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c414a310 con 0x7f36c415cae0 2023-06-28T20:08:03.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.507+0000 7f36ba7fc700 1 -- 172.21.15.43:0/3044877392 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f36bc012580 con 0x7f36c415cae0 2023-06-28T20:08:03.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.507+0000 7f36ba7fc700 1 --2- 172.21.15.43:0/3044877392 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f36a806ace0 0x7f36a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:03.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.507+0000 7f36ba7fc700 1 -- 172.21.15.43:0/3044877392 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f36bc0897a0 con 0x7f36c415cae0 2023-06-28T20:08:03.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.507+0000 7f36c9cf3700 1 --2- 172.21.15.43:0/3044877392 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f36a806ace0 0x7f36a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:03.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.508+0000 7f36c9cf3700 1 --2- 172.21.15.43:0/3044877392 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f36a806ace0 0x7f36a806d190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f36b40010e0 tx=0x7f36b4007d90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:03.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.512+0000 7f36ba7fc700 1 -- 172.21.15.43:0/3044877392 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f36c414a310 con 0x7f36c415cae0 2023-06-28T20:08:03.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.709+0000 7f36cbf57700 1 -- 172.21.15.43:0/3044877392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f36c414a310 con 0x7f36c415cae0 2023-06-28T20:08:03.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.710+0000 7f36ba7fc700 1 -- 172.21.15.43:0/3044877392 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f36c414a310 con 0x7f36c415cae0 2023-06-28T20:08:03.714 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-06-28T20:08:03.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.712+0000 7f36cbf57700 1 -- 172.21.15.43:0/3044877392 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f36a806ace0 msgr2=0x7f36a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:03.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.712+0000 7f36cbf57700 1 --2- 172.21.15.43:0/3044877392 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f36a806ace0 0x7f36a806d190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f36b40010e0 tx=0x7f36b4007d90 comp rx=0 tx=0).stop 2023-06-28T20:08:03.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.712+0000 7f36cbf57700 1 -- 172.21.15.43:0/3044877392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36c415cae0 msgr2=0x7f36c41488c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:03.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.712+0000 7f36cbf57700 1 --2- 172.21.15.43:0/3044877392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36c415cae0 0x7f36c41488c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00cc10 tx=0x7f36bc00cf20 comp rx=0 tx=0).stop 2023-06-28T20:08:03.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.712+0000 7f36cbf57700 1 -- 172.21.15.43:0/3044877392 shutdown_connections 2023-06-28T20:08:03.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.712+0000 7f36cbf57700 1 --2- 172.21.15.43:0/3044877392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36c415cae0 0x7f36c41488c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.712+0000 7f36cbf57700 1 --2- 172.21.15.43:0/3044877392 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36c406d310 0x7f36c414c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.712+0000 7f36cbf57700 1 --2- 172.21.15.43:0/3044877392 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f36a806ace0 0x7f36a806d190 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.712+0000 7f36cbf57700 1 --2- 172.21.15.43:0/3044877392 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36c40cfe00 0x7f36c41f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.712+0000 7f36cbf57700 1 -- 172.21.15.43:0/3044877392 >> 172.21.15.43:0/3044877392 conn(0x7f36c40664b0 msgr2=0x7f36c40c7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:03.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.712+0000 7f36cbf57700 1 -- 172.21.15.43:0/3044877392 shutdown_connections 2023-06-28T20:08:03.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.712+0000 7f36cbf57700 1 -- 172.21.15.43:0/3044877392 wait complete. 2023-06-28T20:08:03.728 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-28T20:08:03.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.850+0000 7f8c4aac4700 1 Processor -- start 2023-06-28T20:08:03.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.850+0000 7f8c4aac4700 1 -- start start 2023-06-28T20:08:03.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.851+0000 7f8c4aac4700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c440ceee0 0x7f8c4406d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:03.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.851+0000 7f8c4aac4700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8c440afdf0 0x7f8c4406d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:03.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.851+0000 7f8c4aac4700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c440ae950 con 0x7f8c4415a5f0 2023-06-28T20:08:03.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.851+0000 7f8c4aac4700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c4415a9c0 con 0x7f8c440afdf0 2023-06-28T20:08:03.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.851+0000 7f8c4aac4700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c440add40 con 0x7f8c440ceee0 2023-06-28T20:08:03.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.851+0000 7f8c49061700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8c440afdf0 0x7f8c4406d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:03.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.851+0000 7f8c43fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c440ceee0 0x7f8c4406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:03.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.851+0000 7f8c49061700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f8c440afdf0 0x7f8c4406d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:40494/0 (socket says 172.21.15.43:40494) 2023-06-28T20:08:03.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.851+0000 7f8c49061700 1 -- 172.21.15.43:0/3568137751 learned_addr learned my addr 172.21.15.43:0/3568137751 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:03.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.851+0000 7f8c48860700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f8c4415a5f0 0x7f8c440ce7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:42736/0 (socket says 172.21.15.43:42736) 2023-06-28T20:08:03.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.852+0000 7f8c49061700 1 -- 172.21.15.43:0/3568137751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c440ceee0 msgr2=0x7f8c4406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:03.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.852+0000 7f8c49061700 1 --2- 172.21.15.43:0/3568137751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c440ceee0 0x7f8c4406d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.852+0000 7f8c49061700 1 -- 172.21.15.43:0/3568137751 >> v1:172.21.15.178:6789/0 conn(0x7f8c4415a5f0 legacy=0x7f8c440ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:03.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.852+0000 7f8c49061700 1 -- 172.21.15.43:0/3568137751 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c440ae020 con 0x7f8c440afdf0 2023-06-28T20:08:03.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.852+0000 7f8c49061700 1 --2- 172.21.15.43:0/3568137751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c440afdf0 0x7f8c4406d850 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009e50 tx=0x7f8c3c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0f5528a90296dd4 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:03.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.852+0000 7f8c437fe700 1 -- 172.21.15.43:0/3568137751 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c00e070 con 0x7f8c440afdf0 2023-06-28T20:08:03.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.852+0000 7f8c437fe700 1 -- 172.21.15.43:0/3568137751 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3c00b910 con 0x7f8c440afdf0 2023-06-28T20:08:03.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.852+0000 7f8c437fe700 1 -- 172.21.15.43:0/3568137751 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c00bcc0 con 0x7f8c440afdf0 2023-06-28T20:08:03.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.852+0000 7f8c4aac4700 1 -- 172.21.15.43:0/3568137751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c440afdf0 msgr2=0x7f8c4406d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:03.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.852+0000 7f8c4aac4700 1 --2- 172.21.15.43:0/3568137751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c440afdf0 0x7f8c4406d850 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009e50 tx=0x7f8c3c00b330 comp rx=0 tx=0).stop 2023-06-28T20:08:03.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.853+0000 7f8c4aac4700 1 -- 172.21.15.43:0/3568137751 shutdown_connections 2023-06-28T20:08:03.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.853+0000 7f8c4aac4700 1 --2- 172.21.15.43:0/3568137751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c440afdf0 0x7f8c4406d850 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.853+0000 7f8c4aac4700 1 --2- 172.21.15.43:0/3568137751 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c440ceee0 0x7f8c4406d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.853+0000 7f8c4aac4700 1 -- 172.21.15.43:0/3568137751 >> 172.21.15.43:0/3568137751 conn(0x7f8c440664b0 msgr2=0x7f8c440d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:03.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.853+0000 7f8c4aac4700 1 -- 172.21.15.43:0/3568137751 shutdown_connections 2023-06-28T20:08:03.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.853+0000 7f8c4aac4700 1 -- 172.21.15.43:0/3568137751 wait complete. 2023-06-28T20:08:03.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.854+0000 7f8c4aac4700 1 Processor -- start 2023-06-28T20:08:03.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.854+0000 7f8c4aac4700 1 -- start start 2023-06-28T20:08:03.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.855+0000 7f8c4aac4700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c440afdf0 0x7f8c4414a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:03.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.855+0000 7f8c4aac4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c440ceee0 0x7f8c4415d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:03.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.855+0000 7f8c4aac4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c4415a5f0 0x7f8c4415add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:03.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.855+0000 7f8c4aac4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c440b30d0 con 0x7f8c4415a5f0 2023-06-28T20:08:03.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.855+0000 7f8c4aac4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4414a670 con 0x7f8c440ceee0 2023-06-28T20:08:03.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.855+0000 7f8c4aac4700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4414ab90 con 0x7f8c440afdf0 2023-06-28T20:08:03.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.855+0000 7f8c49061700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c4415a5f0 0x7f8c4415add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:03.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.855+0000 7f8c49061700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c4415a5f0 0x7f8c4415add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54436/0 (socket says 172.21.15.43:54436) 2023-06-28T20:08:03.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.855+0000 7f8c49061700 1 -- 172.21.15.43:0/1532874430 learned_addr learned my addr 172.21.15.43:0/1532874430 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:03.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.855+0000 7f8c48860700 1 --2- 172.21.15.43:0/1532874430 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c440afdf0 0x7f8c4414a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:03.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.855+0000 7f8c43fff700 1 --2- 172.21.15.43:0/1532874430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c440ceee0 0x7f8c4415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:03.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.855+0000 7f8c49061700 1 -- 172.21.15.43:0/1532874430 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c440afdf0 msgr2=0x7f8c4414a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:03.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.855+0000 7f8c49061700 1 --2- 172.21.15.43:0/1532874430 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c440afdf0 0x7f8c4414a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.855+0000 7f8c49061700 1 -- 172.21.15.43:0/1532874430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c440ceee0 msgr2=0x7f8c4415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:03.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.855+0000 7f8c49061700 1 --2- 172.21.15.43:0/1532874430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c440ceee0 0x7f8c4415d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.855+0000 7f8c49061700 1 -- 172.21.15.43:0/1532874430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c009b30 con 0x7f8c4415a5f0 2023-06-28T20:08:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.855+0000 7f8c49061700 1 --2- 172.21.15.43:0/1532874430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c4415a5f0 0x7f8c4415add0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00b680 tx=0x7f8c3c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.856+0000 7f8c417fa700 1 -- 172.21.15.43:0/1532874430 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c0048b0 con 0x7f8c4415a5f0 2023-06-28T20:08:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.856+0000 7f8c417fa700 1 -- 172.21.15.43:0/1532874430 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3c0092e0 con 0x7f8c4415a5f0 2023-06-28T20:08:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.856+0000 7f8c417fa700 1 -- 172.21.15.43:0/1532874430 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c01c680 con 0x7f8c4415a5f0 2023-06-28T20:08:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.856+0000 7f8c4aac4700 1 -- 172.21.15.43:0/1532874430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c440b2450 con 0x7f8c4415a5f0 2023-06-28T20:08:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.856+0000 7f8c4aac4700 1 -- 172.21.15.43:0/1532874430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c440b2a10 con 0x7f8c4415a5f0 2023-06-28T20:08:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.857+0000 7f8c4aac4700 1 -- 172.21.15.43:0/1532874430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c440b3210 con 0x7f8c4415a5f0 2023-06-28T20:08:03.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.857+0000 7f8c417fa700 1 -- 172.21.15.43:0/1532874430 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8c3c01c7e0 con 0x7f8c4415a5f0 2023-06-28T20:08:03.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.857+0000 7f8c417fa700 1 --2- 172.21.15.43:0/1532874430 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f8c2806ace0 0x7f8c2806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:03.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.858+0000 7f8c417fa700 1 -- 172.21.15.43:0/1532874430 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8c3c08ac00 con 0x7f8c4415a5f0 2023-06-28T20:08:03.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.858+0000 7f8c48860700 1 --2- 172.21.15.43:0/1532874430 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f8c2806ace0 0x7f8c2806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:03.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.858+0000 7f8c48860700 1 --2- 172.21.15.43:0/1532874430 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f8c2806ace0 0x7f8c2806d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c38000fb0 tx=0x7f8c38000bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:03.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:03.862+0000 7f8c417fa700 1 -- 172.21.15.43:0/1532874430 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8c440b3210 con 0x7f8c4415a5f0 2023-06-28T20:08:04.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.078+0000 7f8c4aac4700 1 -- 172.21.15.43:0/1532874430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8c440b3210 con 0x7f8c4415a5f0 2023-06-28T20:08:04.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.078+0000 7f8c417fa700 1 -- 172.21.15.43:0/1532874430 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8c440b3210 con 0x7f8c4415a5f0 2023-06-28T20:08:04.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.080+0000 7f8c4aac4700 1 -- 172.21.15.43:0/1532874430 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f8c2806ace0 msgr2=0x7f8c2806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:04.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.080+0000 7f8c4aac4700 1 --2- 172.21.15.43:0/1532874430 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f8c2806ace0 0x7f8c2806d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c38000fb0 tx=0x7f8c38000bc0 comp rx=0 tx=0).stop 2023-06-28T20:08:04.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.080+0000 7f8c4aac4700 1 -- 172.21.15.43:0/1532874430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c4415a5f0 msgr2=0x7f8c4415add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:04.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.080+0000 7f8c4aac4700 1 --2- 172.21.15.43:0/1532874430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c4415a5f0 0x7f8c4415add0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c00b680 tx=0x7f8c3c004ad0 comp rx=0 tx=0).stop 2023-06-28T20:08:04.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.080+0000 7f8c4aac4700 1 -- 172.21.15.43:0/1532874430 shutdown_connections 2023-06-28T20:08:04.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.080+0000 7f8c4aac4700 1 --2- 172.21.15.43:0/1532874430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c4415a5f0 0x7f8c4415add0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:04.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.080+0000 7f8c4aac4700 1 --2- 172.21.15.43:0/1532874430 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f8c2806ace0 0x7f8c2806d190 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:04.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.080+0000 7f8c4aac4700 1 --2- 172.21.15.43:0/1532874430 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f8c440afdf0 0x7f8c4414a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:04.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.080+0000 7f8c4aac4700 1 --2- 172.21.15.43:0/1532874430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c440ceee0 0x7f8c4415d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:04.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.080+0000 7f8c4aac4700 1 -- 172.21.15.43:0/1532874430 >> 172.21.15.43:0/1532874430 conn(0x7f8c440664b0 msgr2=0x7f8c44157080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:04.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.080+0000 7f8c4aac4700 1 -- 172.21.15.43:0/1532874430 shutdown_connections 2023-06-28T20:08:04.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.080+0000 7f8c4aac4700 1 -- 172.21.15.43:0/1532874430 wait complete. 2023-06-28T20:08:04.086 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2023-06-28T20:08:04.097 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-06-28T20:08:04.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:04.098 DEBUG:teuthology.orchestra.run.smithi178:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-06-28T20:08:04.144 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi178.front.sepia.ceph.com 2023-06-28T20:08:04.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:04.145 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-06-28T20:08:04.164 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-06-28T20:08:04.165 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-28T20:08:04.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.287+0000 7f11c115e700 1 Processor -- start 2023-06-28T20:08:04.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.287+0000 7f11c115e700 1 -- start start 2023-06-28T20:08:04.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.288+0000 7f11c115e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f11bc0cbd40 0x7f11bc06b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:04.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.288+0000 7f11c115e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11bc06b9a0 con 0x7f11bc06b540 2023-06-28T20:08:04.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.288+0000 7f11c115e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11bc0ccf10 con 0x7f11bc0d0630 2023-06-28T20:08:04.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.288+0000 7f11c115e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11bc15cd80 con 0x7f11bc0cbd40 2023-06-28T20:08:04.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.288+0000 7f11ba59c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f11bc0cbd40 0x7f11bc06b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:04.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.288+0000 7f11bb59e700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f11bc06b540 0x7f11bc0ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43630/0 (socket says 172.21.15.43:43630) 2023-06-28T20:08:04.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.288+0000 7f11bb59e700 1 -- 172.21.15.43:0/856864018 learned_addr learned my addr 172.21.15.43:0/856864018 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:04.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.288+0000 7f11ba59c700 1 -- 172.21.15.43:0/856864018 >> v1:172.21.15.178:6789/0 conn(0x7f11bc0d0630 legacy=0x7f11bc0d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:04.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.288+0000 7f11ba59c700 1 -- 172.21.15.43:0/856864018 >> v1:172.21.15.43:6789/0 conn(0x7f11bc06b540 legacy=0x7f11bc0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:04.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.288+0000 7f11ba59c700 1 -- 172.21.15.43:0/856864018 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11bc0d0a00 con 0x7f11bc0cbd40 2023-06-28T20:08:04.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.288+0000 7f11ba59c700 1 --2- 172.21.15.43:0/856864018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11bc0cbd40 0x7f11bc06b000 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009870 tx=0x7f11ac009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26541dcb9f9069f0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:04.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.288+0000 7f11b9d9b700 1 -- 172.21.15.43:0/856864018 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac004230 con 0x7f11bc0cbd40 2023-06-28T20:08:04.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.289+0000 7f11b9d9b700 1 -- 172.21.15.43:0/856864018 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ac004390 con 0x7f11bc0cbd40 2023-06-28T20:08:04.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.289+0000 7f11b9d9b700 1 -- 172.21.15.43:0/856864018 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac0036a0 con 0x7f11bc0cbd40 2023-06-28T20:08:04.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.289+0000 7f11c115e700 1 -- 172.21.15.43:0/856864018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11bc0cbd40 msgr2=0x7f11bc06b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:04.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.289+0000 7f11c115e700 1 --2- 172.21.15.43:0/856864018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11bc0cbd40 0x7f11bc06b000 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009870 tx=0x7f11ac009b80 comp rx=0 tx=0).stop 2023-06-28T20:08:04.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.289+0000 7f11c115e700 1 -- 172.21.15.43:0/856864018 shutdown_connections 2023-06-28T20:08:04.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.289+0000 7f11c115e700 1 --2- 172.21.15.43:0/856864018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11bc0cbd40 0x7f11bc06b000 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:04.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.289+0000 7f11c115e700 1 -- 172.21.15.43:0/856864018 >> 172.21.15.43:0/856864018 conn(0x7f11bc0664b0 msgr2=0x7f11bc09a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:04.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.289+0000 7f11c115e700 1 -- 172.21.15.43:0/856864018 shutdown_connections 2023-06-28T20:08:04.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.289+0000 7f11c115e700 1 -- 172.21.15.43:0/856864018 wait complete. 2023-06-28T20:08:04.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.291+0000 7f11c115e700 1 Processor -- start 2023-06-28T20:08:04.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.291+0000 7f11c115e700 1 -- start start 2023-06-28T20:08:04.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.292+0000 7f11c115e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11bc06b540 0x7f11bc0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:04.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.292+0000 7f11c115e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11bc0cbd40 0x7f11bc06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:04.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.292+0000 7f11c115e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11bc0d0630 0x7f11bc06f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:04.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.292+0000 7f11c115e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc0b5380 con 0x7f11bc0d0630 2023-06-28T20:08:04.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.292+0000 7f11c115e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc0c9650 con 0x7f11bc06b540 2023-06-28T20:08:04.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.292+0000 7f11c115e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc0c9930 con 0x7f11bc0cbd40 2023-06-28T20:08:04.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.292+0000 7f11bb59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11bc0d0630 0x7f11bc06f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:04.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.292+0000 7f11bb59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11bc0d0630 0x7f11bc06f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54450/0 (socket says 172.21.15.43:54450) 2023-06-28T20:08:04.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.292+0000 7f11bb59e700 1 -- 172.21.15.43:0/2333374573 learned_addr learned my addr 172.21.15.43:0/2333374573 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.292+0000 7f11ba59c700 1 --2- 172.21.15.43:0/2333374573 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11bc0cbd40 0x7f11bc06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.292+0000 7f11bad9d700 1 --2- 172.21.15.43:0/2333374573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11bc06b540 0x7f11bc0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.292+0000 7f11bb59e700 1 -- 172.21.15.43:0/2333374573 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11bc0cbd40 msgr2=0x7f11bc06ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.292+0000 7f11bb59e700 1 --2- 172.21.15.43:0/2333374573 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11bc0cbd40 0x7f11bc06ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.292+0000 7f11bb59e700 1 -- 172.21.15.43:0/2333374573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11bc06b540 msgr2=0x7f11bc0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:04.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.292+0000 7f11bb59e700 1 --2- 172.21.15.43:0/2333374573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11bc06b540 0x7f11bc0c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:04.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.292+0000 7f11bb59e700 1 -- 172.21.15.43:0/2333374573 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ac0094a0 con 0x7f11bc0d0630 2023-06-28T20:08:04.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.292+0000 7f11bad9d700 1 --2- 172.21.15.43:0/2333374573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11bc06b540 0x7f11bc0c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:04.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.292+0000 7f11bb59e700 1 --2- 172.21.15.43:0/2333374573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11bc0d0630 0x7f11bc06f3b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f11b400ca80 tx=0x7f11b400cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:04.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.292+0000 7f11a37fe700 1 -- 172.21.15.43:0/2333374573 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b400a9e0 con 0x7f11bc0d0630 2023-06-28T20:08:04.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.292+0000 7f11a37fe700 1 -- 172.21.15.43:0/2333374573 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b400ab40 con 0x7f11bc0d0630 2023-06-28T20:08:04.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.292+0000 7f11a37fe700 1 -- 172.21.15.43:0/2333374573 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b4012400 con 0x7f11bc0d0630 2023-06-28T20:08:04.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.292+0000 7f11c115e700 1 -- 172.21.15.43:0/2333374573 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11bc0c9c10 con 0x7f11bc0d0630 2023-06-28T20:08:04.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.293+0000 7f11c115e700 1 -- 172.21.15.43:0/2333374573 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11bc0b4f30 con 0x7f11bc0d0630 2023-06-28T20:08:04.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.293+0000 7f11c115e700 1 -- 172.21.15.43:0/2333374573 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11bc0728b0 con 0x7f11bc0d0630 2023-06-28T20:08:04.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.294+0000 7f11a37fe700 1 -- 172.21.15.43:0/2333374573 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f11b401b3b0 con 0x7f11bc0d0630 2023-06-28T20:08:04.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.294+0000 7f11a37fe700 1 --2- 172.21.15.43:0/2333374573 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f11a806ace0 0x7f11a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.294+0000 7f11a37fe700 1 -- 172.21.15.43:0/2333374573 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f11b4088ef0 con 0x7f11bc0d0630 2023-06-28T20:08:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.294+0000 7f11bad9d700 1 --2- 172.21.15.43:0/2333374573 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f11a806ace0 0x7f11a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.295+0000 7f11bad9d700 1 --2- 172.21.15.43:0/2333374573 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f11a806ace0 0x7f11a806d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f11b0007fd0 tx=0x7f11b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:04.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.299+0000 7f11a37fe700 1 -- 172.21.15.43:0/2333374573 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f11b40540e0 con 0x7f11bc0d0630 2023-06-28T20:08:04.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.542+0000 7f11c115e700 1 -- 172.21.15.43:0/2333374573 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11bc0cc890 con 0x7f11bc0d0630 2023-06-28T20:08:04.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.543+0000 7f11a37fe700 1 -- 172.21.15.43:0/2333374573 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4665 (secure 0 0 0) 0x7f11b4057700 con 0x7f11bc0d0630 2023-06-28T20:08:04.547 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:04.547 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4033115951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4033115951},{"type":"v1","addr":"172.21.15.178:6835","nonce":4033115951}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/65569869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":65569869},{"type":"v1","addr":"172.21.15.43:6837","nonce":65569869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/1016213057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016213057},{"type":"v1","addr":"172.21.15.178:6834","nonce":1016213057}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-28T20:07:20.788417+0000","modified":"2023-06-28T20:07:24.388714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/3514544484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3514544484},{"type":"v1","addr":"172.21.15.43:6835","nonce":3514544484}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-28T20:08:04.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.544+0000 7f11c115e700 1 -- 172.21.15.43:0/2333374573 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f11a806ace0 msgr2=0x7f11a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:04.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.544+0000 7f11c115e700 1 --2- 172.21.15.43:0/2333374573 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f11a806ace0 0x7f11a806d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f11b0007fd0 tx=0x7f11b0009040 comp rx=0 tx=0).stop 2023-06-28T20:08:04.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.544+0000 7f11c115e700 1 -- 172.21.15.43:0/2333374573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11bc0d0630 msgr2=0x7f11bc06f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:04.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.544+0000 7f11c115e700 1 --2- 172.21.15.43:0/2333374573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11bc0d0630 0x7f11bc06f3b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f11b400ca80 tx=0x7f11b400cd90 comp rx=0 tx=0).stop 2023-06-28T20:08:04.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.544+0000 7f11c115e700 1 -- 172.21.15.43:0/2333374573 shutdown_connections 2023-06-28T20:08:04.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.544+0000 7f11c115e700 1 --2- 172.21.15.43:0/2333374573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f11bc0d0630 0x7f11bc06f3b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:04.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.544+0000 7f11c115e700 1 --2- 172.21.15.43:0/2333374573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11bc06b540 0x7f11bc0c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:04.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.545+0000 7f11c115e700 1 --2- 172.21.15.43:0/2333374573 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f11a806ace0 0x7f11a806d190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:04.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.545+0000 7f11c115e700 1 --2- 172.21.15.43:0/2333374573 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f11bc0cbd40 0x7f11bc06ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:04.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.545+0000 7f11c115e700 1 -- 172.21.15.43:0/2333374573 >> 172.21.15.43:0/2333374573 conn(0x7f11bc0664b0 msgr2=0x7f11bc0ad930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:04.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.545+0000 7f11c115e700 1 -- 172.21.15.43:0/2333374573 shutdown_connections 2023-06-28T20:08:04.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.545+0000 7f11c115e700 1 -- 172.21.15.43:0/2333374573 wait complete. 2023-06-28T20:08:04.550 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 7 2023-06-28T20:08:04.561 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:08:04.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.681+0000 7f9576482700 1 Processor -- start 2023-06-28T20:08:04.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.681+0000 7f9576482700 1 -- start start 2023-06-28T20:08:04.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.682+0000 7f9576482700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95700ceb60 0x7f95700c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:04.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.682+0000 7f9576482700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95700c9f50 con 0x7f9570071580 2023-06-28T20:08:04.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.682+0000 7f9576482700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95700cb370 con 0x7f9570071950 2023-06-28T20:08:04.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.682+0000 7f9576482700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95700cefe0 con 0x7f95700ceb60 2023-06-28T20:08:04.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.682+0000 7f956ffff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9570071580 0x7f9570069ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43640/0 (socket says 172.21.15.43:43640) 2023-06-28T20:08:04.687 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.682+0000 7f956ffff700 1 -- 172.21.15.43:0/3260304745 learned_addr learned my addr 172.21.15.43:0/3260304745 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:04.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.682+0000 7f9574a1f700 1 --2- 172.21.15.43:0/3260304745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95700ceb60 0x7f95700c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:04.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.682+0000 7f956effd700 1 -- 172.21.15.43:0/3260304745 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1820535912 0 0) 0x7f9560001110 con 0x7f9570071580 2023-06-28T20:08:04.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.682+0000 7f956effd700 1 -- 172.21.15.43:0/3260304745 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95580035b0 con 0x7f9570071580 2023-06-28T20:08:04.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.683+0000 7f956effd700 1 -- 172.21.15.43:0/3260304745 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2789115591 0 0) 0x7f95700cb370 con 0x7f9570071950 2023-06-28T20:08:04.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.683+0000 7f956effd700 1 -- 172.21.15.43:0/3260304745 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9560001110 con 0x7f9570071950 2023-06-28T20:08:04.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.683+0000 7f956effd700 1 -- 172.21.15.43:0/3260304745 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 253459943 0 0) 0x7f95580035b0 con 0x7f9570071580 2023-06-28T20:08:04.688 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.683+0000 7f9574a1f700 1 -- 172.21.15.43:0/3260304745 >> v1:172.21.15.178:6789/0 conn(0x7f9570071950 legacy=0x7f95700cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:04.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.683+0000 7f9574a1f700 1 -- 172.21.15.43:0/3260304745 >> v1:172.21.15.43:6789/0 conn(0x7f9570071580 legacy=0x7f9570069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:04.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.683+0000 7f9574a1f700 1 -- 172.21.15.43:0/3260304745 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95700cbbb0 con 0x7f95700ceb60 2023-06-28T20:08:04.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.683+0000 7f9574a1f700 1 --2- 172.21.15.43:0/3260304745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95700ceb60 0x7f95700c9a10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9568009dc0 tx=0x7f956800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d6349f04d8ad769 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:04.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.683+0000 7f956effd700 1 -- 172.21.15.43:0/3260304745 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956800e070 con 0x7f95700ceb60 2023-06-28T20:08:04.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.683+0000 7f956effd700 1 -- 172.21.15.43:0/3260304745 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9568003a20 con 0x7f95700ceb60 2023-06-28T20:08:04.689 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.683+0000 7f956effd700 1 -- 172.21.15.43:0/3260304745 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9560001800 con 0x7f95700ceb60 2023-06-28T20:08:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.683+0000 7f9576482700 1 -- 172.21.15.43:0/3260304745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95700ceb60 msgr2=0x7f95700c9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.683+0000 7f9576482700 1 --2- 172.21.15.43:0/3260304745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95700ceb60 0x7f95700c9a10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9568009dc0 tx=0x7f956800b330 comp rx=0 tx=0).stop 2023-06-28T20:08:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.684+0000 7f9576482700 1 -- 172.21.15.43:0/3260304745 shutdown_connections 2023-06-28T20:08:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.684+0000 7f9576482700 1 --2- 172.21.15.43:0/3260304745 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f95700ceb60 0x7f95700c9a10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:04.691 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.684+0000 7f9576482700 1 -- 172.21.15.43:0/3260304745 >> 172.21.15.43:0/3260304745 conn(0x7f957006c170 msgr2=0x7f957006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.684+0000 7f9576482700 1 -- 172.21.15.43:0/3260304745 shutdown_connections 2023-06-28T20:08:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.684+0000 7f9576482700 1 -- 172.21.15.43:0/3260304745 wait complete. 2023-06-28T20:08:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.685+0000 7f9576482700 1 Processor -- start 2023-06-28T20:08:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.685+0000 7f9576482700 1 -- start start 2023-06-28T20:08:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.686+0000 7f9576482700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9570071580 0x7f95701e1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.686+0000 7f9576482700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9570071950 0x7f957014a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.686+0000 7f9576482700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95700ceb60 0x7f95700c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:04.692 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.686+0000 7f9576482700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95701e1840 con 0x7f95700ceb60 2023-06-28T20:08:04.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.686+0000 7f9576482700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95701f2de0 con 0x7f9570071950 2023-06-28T20:08:04.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.686+0000 7f9576482700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95701f30c0 con 0x7f9570071580 2023-06-28T20:08:04.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.686+0000 7f9574a1f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95700ceb60 0x7f95700c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:04.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.686+0000 7f9574a1f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95700ceb60 0x7f95700c4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54464/0 (socket says 172.21.15.43:54464) 2023-06-28T20:08:04.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.686+0000 7f9574a1f700 1 -- 172.21.15.43:0/70662263 learned_addr learned my addr 172.21.15.43:0/70662263 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:04.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.687+0000 7f956ffff700 1 --2- 172.21.15.43:0/70662263 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9570071580 0x7f95701e1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:04.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.687+0000 7f956f7fe700 1 --2- 172.21.15.43:0/70662263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9570071950 0x7f957014a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.687+0000 7f9574a1f700 1 -- 172.21.15.43:0/70662263 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9570071580 msgr2=0x7f95701e1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.687+0000 7f9574a1f700 1 --2- 172.21.15.43:0/70662263 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9570071580 0x7f95701e1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.687+0000 7f9574a1f700 1 -- 172.21.15.43:0/70662263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9570071950 msgr2=0x7f957014a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.687+0000 7f9574a1f700 1 --2- 172.21.15.43:0/70662263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9570071950 0x7f957014a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.687+0000 7f9574a1f700 1 -- 172.21.15.43:0/70662263 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9568009a70 con 0x7f95700ceb60 2023-06-28T20:08:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.687+0000 7f9574a1f700 1 --2- 172.21.15.43:0/70662263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95700ceb60 0x7f95700c4df0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9568009d90 tx=0x7f9568004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.687+0000 7f956cff9700 1 -- 172.21.15.43:0/70662263 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95680042c0 con 0x7f95700ceb60 2023-06-28T20:08:04.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.687+0000 7f9576482700 1 -- 172.21.15.43:0/70662263 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95701f33a0 con 0x7f95700ceb60 2023-06-28T20:08:04.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.687+0000 7f956cff9700 1 -- 172.21.15.43:0/70662263 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9568004460 con 0x7f95700ceb60 2023-06-28T20:08:04.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.687+0000 7f956cff9700 1 -- 172.21.15.43:0/70662263 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9568004810 con 0x7f95700ceb60 2023-06-28T20:08:04.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.687+0000 7f9576482700 1 -- 172.21.15.43:0/70662263 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957014e060 con 0x7f95700ceb60 2023-06-28T20:08:04.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.688+0000 7f9576482700 1 -- 172.21.15.43:0/70662263 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95700ca6b0 con 0x7f95700ceb60 2023-06-28T20:08:04.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.688+0000 7f956cff9700 1 -- 172.21.15.43:0/70662263 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f956801c460 con 0x7f95700ceb60 2023-06-28T20:08:04.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.689+0000 7f956cff9700 1 --2- 172.21.15.43:0/70662263 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f955c06ace0 0x7f955c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:04.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.689+0000 7f956cff9700 1 -- 172.21.15.43:0/70662263 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f956808a950 con 0x7f95700ceb60 2023-06-28T20:08:04.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.689+0000 7f956ffff700 1 --2- 172.21.15.43:0/70662263 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f955c06ace0 0x7f955c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:04.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.689+0000 7f956ffff700 1 --2- 172.21.15.43:0/70662263 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f955c06ace0 0x7f955c06d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9560000f60 tx=0x7f9560001e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:04.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.693+0000 7f956cff9700 1 -- 172.21.15.43:0/70662263 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9568055ac0 con 0x7f95700ceb60 2023-06-28T20:08:04.895 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.891+0000 7f9576482700 1 -- 172.21.15.43:0/70662263 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95700728b0 con 0x7f95700ceb60 2023-06-28T20:08:04.896 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.892+0000 7f956cff9700 1 -- 172.21.15.43:0/70662263 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18221 (secure 0 0 0) 0x7f95680590e0 con 0x7f95700ceb60 2023-06-28T20:08:04.897 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:04.897 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:07:22.377248+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:07:18.624759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-28T20:07:19.760017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:08:04.898 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.894+0000 7f9576482700 1 -- 172.21.15.43:0/70662263 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f955c06ace0 msgr2=0x7f955c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:04.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.894+0000 7f9576482700 1 --2- 172.21.15.43:0/70662263 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f955c06ace0 0x7f955c06d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9560000f60 tx=0x7f9560001e70 comp rx=0 tx=0).stop 2023-06-28T20:08:04.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.894+0000 7f9576482700 1 -- 172.21.15.43:0/70662263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95700ceb60 msgr2=0x7f95700c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:04.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.894+0000 7f9576482700 1 --2- 172.21.15.43:0/70662263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95700ceb60 0x7f95700c4df0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9568009d90 tx=0x7f9568004ad0 comp rx=0 tx=0).stop 2023-06-28T20:08:04.899 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.894+0000 7f9576482700 1 -- 172.21.15.43:0/70662263 shutdown_connections 2023-06-28T20:08:04.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.894+0000 7f9576482700 1 --2- 172.21.15.43:0/70662263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95700ceb60 0x7f95700c4df0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:04.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.894+0000 7f9576482700 1 --2- 172.21.15.43:0/70662263 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f955c06ace0 0x7f955c06d190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:04.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.894+0000 7f9576482700 1 --2- 172.21.15.43:0/70662263 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9570071580 0x7f95701e1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:04.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.894+0000 7f9576482700 1 --2- 172.21.15.43:0/70662263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9570071950 0x7f957014a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:04.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.894+0000 7f9576482700 1 -- 172.21.15.43:0/70662263 >> 172.21.15.43:0/70662263 conn(0x7f957006c170 msgr2=0x7f95700cbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:04.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.895+0000 7f9576482700 1 -- 172.21.15.43:0/70662263 shutdown_connections 2023-06-28T20:08:04.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:04.895+0000 7f9576482700 1 -- 172.21.15.43:0/70662263 wait complete. 2023-06-28T20:08:04.911 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-28T20:07:13.580969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-28T20:07:14.038358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-28T20:07:18.624759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-28T20:07:19.760017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-28T20:08:04.912 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-28T20:08:05.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.036+0000 7fdcb320d700 1 Processor -- start 2023-06-28T20:08:05.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.036+0000 7fdcb320d700 1 -- start start 2023-06-28T20:08:05.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.037+0000 7fdcb320d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdcac15a5f0 0x7fdcac0ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:05.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.037+0000 7fdcb320d700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdcac06e320 0x7fdcac0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:05.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.037+0000 7fdcb320d700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcac0ae950 con 0x7fdcac0ad7d0 2023-06-28T20:08:05.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.037+0000 7fdcb320d700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcac15a9c0 con 0x7fdcac06e320 2023-06-28T20:08:05.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.037+0000 7fdcb320d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac0add40 con 0x7fdcac15a5f0 2023-06-28T20:08:05.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.037+0000 7fdcb17aa700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdcac06e320 0x7fdcac0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:05.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.037+0000 7fdcb0fa9700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdcac15a5f0 0x7fdcac0ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:05.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.037+0000 7fdcabfff700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fdcac0ad7d0 0x7fdcac0aede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:42780/0 (socket says 172.21.15.43:42780) 2023-06-28T20:08:05.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.037+0000 7fdcabfff700 1 -- 172.21.15.43:0/3935243660 learned_addr learned my addr 172.21.15.43:0/3935243660 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:05.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.038+0000 7fdcab7fe700 1 -- 172.21.15.43:0/3935243660 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2782293282 0 0) 0x7fdcac0ae950 con 0x7fdcac0ad7d0 2023-06-28T20:08:05.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.038+0000 7fdcab7fe700 1 -- 172.21.15.43:0/3935243660 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc9c0035b0 con 0x7fdcac0ad7d0 2023-06-28T20:08:05.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.038+0000 7fdcb0fa9700 1 -- 172.21.15.43:0/3935243660 >> v2:172.21.15.178:3300/0 conn(0x7fdcac06e320 msgr2=0x7fdcac0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:05.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.038+0000 7fdcb0fa9700 1 --2- 172.21.15.43:0/3935243660 >> v2:172.21.15.178:3300/0 conn(0x7fdcac06e320 0x7fdcac0ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.038+0000 7fdcb0fa9700 1 -- 172.21.15.43:0/3935243660 >> v1:172.21.15.178:6789/0 conn(0x7fdcac0ad7d0 legacy=0x7fdcac0aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:05.043 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.038+0000 7fdcb0fa9700 1 -- 172.21.15.43:0/3935243660 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcac0ae020 con 0x7fdcac15a5f0 2023-06-28T20:08:05.044 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.038+0000 7fdcb0fa9700 1 --2- 172.21.15.43:0/3935243660 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdcac15a5f0 0x7fdcac0ce7d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009820 tx=0x7fdca0009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34db063b5b80f937 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:05.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.038+0000 7fdcab7fe700 1 -- 172.21.15.43:0/3935243660 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca0005e30 con 0x7fdcac15a5f0 2023-06-28T20:08:05.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.038+0000 7fdcab7fe700 1 -- 172.21.15.43:0/3935243660 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca0006030 con 0x7fdcac15a5f0 2023-06-28T20:08:05.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.038+0000 7fdcab7fe700 1 -- 172.21.15.43:0/3935243660 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca00063a0 con 0x7fdcac15a5f0 2023-06-28T20:08:05.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.039+0000 7fdcb320d700 1 -- 172.21.15.43:0/3935243660 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdcac15a5f0 msgr2=0x7fdcac0ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:05.046 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.039+0000 7fdcb320d700 1 --2- 172.21.15.43:0/3935243660 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdcac15a5f0 0x7fdcac0ce7d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009820 tx=0x7fdca0009b30 comp rx=0 tx=0).stop 2023-06-28T20:08:05.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.039+0000 7fdcb320d700 1 -- 172.21.15.43:0/3935243660 shutdown_connections 2023-06-28T20:08:05.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.039+0000 7fdcb320d700 1 --2- 172.21.15.43:0/3935243660 >> v2:172.21.15.178:3300/0 conn(0x7fdcac06e320 0x7fdcac0ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.039+0000 7fdcb320d700 1 --2- 172.21.15.43:0/3935243660 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdcac15a5f0 0x7fdcac0ce7d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.039+0000 7fdcb320d700 1 -- 172.21.15.43:0/3935243660 >> 172.21.15.43:0/3935243660 conn(0x7fdcac0664b0 msgr2=0x7fdcac0d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:05.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.039+0000 7fdcb320d700 1 -- 172.21.15.43:0/3935243660 shutdown_connections 2023-06-28T20:08:05.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.039+0000 7fdcb320d700 1 -- 172.21.15.43:0/3935243660 wait complete. 2023-06-28T20:08:05.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.040+0000 7fdcb320d700 1 Processor -- start 2023-06-28T20:08:05.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.040+0000 7fdcb320d700 1 -- start start 2023-06-28T20:08:05.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.041+0000 7fdcb320d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdcac06e320 0x7fdcac155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:05.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.041+0000 7fdcb320d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcac0ad7d0 0x7fdcac0ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:05.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.041+0000 7fdcb320d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcac15a5f0 0x7fdcac0b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:05.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.041+0000 7fdcb320d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac06b660 con 0x7fdcac0ad7d0 2023-06-28T20:08:05.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.041+0000 7fdcb320d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac06b310 con 0x7fdcac15a5f0 2023-06-28T20:08:05.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.041+0000 7fdcb320d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcac0cd540 con 0x7fdcac06e320 2023-06-28T20:08:05.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.041+0000 7fdcabfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcac0ad7d0 0x7fdcac0ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:05.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.041+0000 7fdcabfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcac0ad7d0 0x7fdcac0ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54474/0 (socket says 172.21.15.43:54474) 2023-06-28T20:08:05.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.041+0000 7fdcabfff700 1 -- 172.21.15.43:0/2771209916 learned_addr learned my addr 172.21.15.43:0/2771209916 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:05.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.041+0000 7fdcb0fa9700 1 --2- 172.21.15.43:0/2771209916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdcac06e320 0x7fdcac155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:05.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.041+0000 7fdcb17aa700 1 --2- 172.21.15.43:0/2771209916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcac15a5f0 0x7fdcac0b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:05.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.041+0000 7fdcabfff700 1 -- 172.21.15.43:0/2771209916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdcac06e320 msgr2=0x7fdcac155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:05.049 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.041+0000 7fdcabfff700 1 --2- 172.21.15.43:0/2771209916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdcac06e320 0x7fdcac155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.041+0000 7fdcabfff700 1 -- 172.21.15.43:0/2771209916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcac15a5f0 msgr2=0x7fdcac0b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:05.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.041+0000 7fdcabfff700 1 --2- 172.21.15.43:0/2771209916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcac15a5f0 0x7fdcac0b0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.041+0000 7fdcabfff700 1 -- 172.21.15.43:0/2771209916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca0009430 con 0x7fdcac0ad7d0 2023-06-28T20:08:05.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.042+0000 7fdcabfff700 1 --2- 172.21.15.43:0/2771209916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcac0ad7d0 0x7fdcac0ccaf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800c4d0 tx=0x7fdc9800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:05.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.042+0000 7fdca97fa700 1 -- 172.21.15.43:0/2771209916 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc98005d70 con 0x7fdcac0ad7d0 2023-06-28T20:08:05.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.042+0000 7fdcb320d700 1 -- 172.21.15.43:0/2771209916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcac0cd820 con 0x7fdcac0ad7d0 2023-06-28T20:08:05.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.042+0000 7fdca97fa700 1 -- 172.21.15.43:0/2771209916 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc980070d0 con 0x7fdcac0ad7d0 2023-06-28T20:08:05.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.042+0000 7fdca97fa700 1 -- 172.21.15.43:0/2771209916 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc98007440 con 0x7fdcac0ad7d0 2023-06-28T20:08:05.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.042+0000 7fdcb320d700 1 -- 172.21.15.43:0/2771209916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcac0cd030 con 0x7fdcac0ad7d0 2023-06-28T20:08:05.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.043+0000 7fdcb320d700 1 -- 172.21.15.43:0/2771209916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcac0cd2b0 con 0x7fdcac0ad7d0 2023-06-28T20:08:05.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.043+0000 7fdca97fa700 1 -- 172.21.15.43:0/2771209916 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdc980075a0 con 0x7fdcac0ad7d0 2023-06-28T20:08:05.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.044+0000 7fdca97fa700 1 --2- 172.21.15.43:0/2771209916 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fdc9006ace0 0x7fdc9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:05.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.044+0000 7fdca97fa700 1 -- 172.21.15.43:0/2771209916 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdc980888c0 con 0x7fdcac0ad7d0 2023-06-28T20:08:05.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.044+0000 7fdcb0fa9700 1 --2- 172.21.15.43:0/2771209916 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fdc9006ace0 0x7fdc9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:05.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.044+0000 7fdcb0fa9700 1 --2- 172.21.15.43:0/2771209916 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fdc9006ace0 0x7fdc9006d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdca0000c00 tx=0x7fdca0008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:05.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.048+0000 7fdca97fa700 1 -- 172.21.15.43:0/2771209916 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdc98053a30 con 0x7fdcac0ad7d0 2023-06-28T20:08:05.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.247+0000 7fdcb320d700 1 -- 172.21.15.43:0/2771209916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdcac0728b0 con 0x7fdcac0ad7d0 2023-06-28T20:08:05.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.247+0000 7fdca97fa700 1 -- 172.21.15.43:0/2771209916 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v21) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdc98057050 con 0x7fdcac0ad7d0 2023-06-28T20:08:05.252 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-06-28T20:08:05.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.249+0000 7fdcb320d700 1 -- 172.21.15.43:0/2771209916 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fdc9006ace0 msgr2=0x7fdc9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:05.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.249+0000 7fdcb320d700 1 --2- 172.21.15.43:0/2771209916 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fdc9006ace0 0x7fdc9006d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdca0000c00 tx=0x7fdca0008070 comp rx=0 tx=0).stop 2023-06-28T20:08:05.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.249+0000 7fdcb320d700 1 -- 172.21.15.43:0/2771209916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcac0ad7d0 msgr2=0x7fdcac0ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:05.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.249+0000 7fdcb320d700 1 --2- 172.21.15.43:0/2771209916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcac0ad7d0 0x7fdcac0ccaf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800c4d0 tx=0x7fdc9800c890 comp rx=0 tx=0).stop 2023-06-28T20:08:05.255 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.249+0000 7fdcb320d700 1 -- 172.21.15.43:0/2771209916 shutdown_connections 2023-06-28T20:08:05.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.249+0000 7fdcb320d700 1 --2- 172.21.15.43:0/2771209916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcac15a5f0 0x7fdcac0b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.249+0000 7fdcb320d700 1 --2- 172.21.15.43:0/2771209916 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fdc9006ace0 0x7fdc9006d190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.249+0000 7fdcb320d700 1 --2- 172.21.15.43:0/2771209916 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdcac06e320 0x7fdcac155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.249+0000 7fdcb320d700 1 --2- 172.21.15.43:0/2771209916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcac0ad7d0 0x7fdcac0ccaf0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.249+0000 7fdcb320d700 1 -- 172.21.15.43:0/2771209916 >> 172.21.15.43:0/2771209916 conn(0x7fdcac0664b0 msgr2=0x7fdcac159340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:05.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.249+0000 7fdcb320d700 1 -- 172.21.15.43:0/2771209916 shutdown_connections 2023-06-28T20:08:05.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.249+0000 7fdcb320d700 1 -- 172.21.15.43:0/2771209916 wait complete. 2023-06-28T20:08:05.265 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-28T20:08:05.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.389+0000 7f1fc193e700 1 Processor -- start 2023-06-28T20:08:05.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.389+0000 7f1fc193e700 1 -- start start 2023-06-28T20:08:05.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.390+0000 7f1fc193e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1fbc06d310 0x7f1fbc06cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:05.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.390+0000 7f1fc193e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fbc0cfe00 0x7f1fbc0cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:05.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.390+0000 7f1fc193e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fbc0d0680 con 0x7f1fbc15cae0 2023-06-28T20:08:05.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.390+0000 7f1fc193e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fbc06bb20 con 0x7f1fbc06d310 2023-06-28T20:08:05.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.390+0000 7f1fc193e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc06be00 con 0x7f1fbc0cfe00 2023-06-28T20:08:05.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.390+0000 7f1fba7fc700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1fbc15cae0 0x7f1fbc06d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43670/0 (socket says 172.21.15.43:43670) 2023-06-28T20:08:05.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.390+0000 7f1fba7fc700 1 -- 172.21.15.43:0/742952408 learned_addr learned my addr 172.21.15.43:0/742952408 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:05.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fbaffd700 1 --2- 172.21.15.43:0/742952408 >> v2:172.21.15.178:3300/0 conn(0x7f1fbc06d310 0x7f1fbc06cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:05.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fbb7fe700 1 --2- 172.21.15.43:0/742952408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fbc0cfe00 0x7f1fbc0cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:05.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fb9ffb700 1 -- 172.21.15.43:0/742952408 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323377032 0 0) 0x7f1fbc0d0680 con 0x7f1fbc15cae0 2023-06-28T20:08:05.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fb9ffb700 1 -- 172.21.15.43:0/742952408 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fa40035b0 con 0x7f1fbc15cae0 2023-06-28T20:08:05.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fb9ffb700 1 -- 172.21.15.43:0/742952408 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2346925543 0 0) 0x7f1fa40035b0 con 0x7f1fbc15cae0 2023-06-28T20:08:05.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fb9ffb700 1 -- 172.21.15.43:0/742952408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fbc0cfe00 msgr2=0x7f1fbc0cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:05.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fb9ffb700 1 --2- 172.21.15.43:0/742952408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fbc0cfe00 0x7f1fbc0cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fb9ffb700 1 -- 172.21.15.43:0/742952408 >> v2:172.21.15.178:3300/0 conn(0x7f1fbc06d310 msgr2=0x7f1fbc06cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:05.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fb9ffb700 1 --2- 172.21.15.43:0/742952408 >> v2:172.21.15.178:3300/0 conn(0x7f1fbc06d310 0x7f1fbc06cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fb9ffb700 1 -- 172.21.15.43:0/742952408 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fbc06c0e0 con 0x7f1fbc15cae0 2023-06-28T20:08:05.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fb9ffb700 1 -- 172.21.15.43:0/742952408 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f1fac001670 con 0x7f1fbc15cae0 2023-06-28T20:08:05.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fb9ffb700 1 -- 172.21.15.43:0/742952408 >> v1:172.21.15.43:6789/0 conn(0x7f1fbc15cae0 legacy=0x7f1fbc06d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:05.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fb9ffb700 1 --2- 172.21.15.43:0/742952408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fbc0cfe00 0x7f1fa4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:05.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fb9ffb700 1 --2- 172.21.15.43:0/742952408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fa4003f80 0x7f1fa4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:05.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fb9ffb700 1 --2- 172.21.15.43:0/742952408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fa4004d90 0x7f1fa4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:05.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fb9ffb700 1 -- 172.21.15.43:0/742952408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc06bb20 con 0x7f1fa4003f80 2023-06-28T20:08:05.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fb9ffb700 1 -- 172.21.15.43:0/742952408 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc06be00 con 0x7f1fa4004d90 2023-06-28T20:08:05.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fb9ffb700 1 -- 172.21.15.43:0/742952408 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa400ba10 con 0x7f1fbc0cfe00 2023-06-28T20:08:05.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fba7fc700 1 --2- 172.21.15.43:0/742952408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fa4003f80 0x7f1fa4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:05.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fbaffd700 1 --2- 172.21.15.43:0/742952408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fbc0cfe00 0x7f1fa4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:05.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fbb7fe700 1 --2- 172.21.15.43:0/742952408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fa4004d90 0x7f1fa4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:05.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fba7fc700 1 -- 172.21.15.43:0/742952408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fbc0cfe00 msgr2=0x7f1fa4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:05.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fba7fc700 1 --2- 172.21.15.43:0/742952408 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fbc0cfe00 0x7f1fa4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fba7fc700 1 -- 172.21.15.43:0/742952408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fa4004d90 msgr2=0x7f1fa4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:05.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fba7fc700 1 --2- 172.21.15.43:0/742952408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fa4004d90 0x7f1fa4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fba7fc700 1 -- 172.21.15.43:0/742952408 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa40035b0 con 0x7f1fa4003f80 2023-06-28T20:08:05.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.391+0000 7f1fba7fc700 1 --2- 172.21.15.43:0/742952408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fa4003f80 0x7f1fa4004850 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00c2f0 tx=0x7f1fa4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca22eb684cbb2ef server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:05.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.392+0000 7f1fb9ffb700 1 -- 172.21.15.43:0/742952408 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac00ce30 con 0x7f1fa4003f80 2023-06-28T20:08:05.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.392+0000 7f1fb9ffb700 1 -- 172.21.15.43:0/742952408 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fac005960 con 0x7f1fa4003f80 2023-06-28T20:08:05.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.392+0000 7f1fb9ffb700 1 -- 172.21.15.43:0/742952408 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac005cf0 con 0x7f1fa4003f80 2023-06-28T20:08:05.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.392+0000 7f1fc193e700 1 -- 172.21.15.43:0/742952408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fa4003f80 msgr2=0x7f1fa4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:05.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.392+0000 7f1fc193e700 1 --2- 172.21.15.43:0/742952408 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fa4003f80 0x7f1fa4004850 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00c2f0 tx=0x7f1fa4005880 comp rx=0 tx=0).stop 2023-06-28T20:08:05.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.392+0000 7f1fbb7fe700 1 -- 172.21.15.43:0/742952408 reap_dead start 2023-06-28T20:08:05.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.392+0000 7f1fc193e700 1 -- 172.21.15.43:0/742952408 shutdown_connections 2023-06-28T20:08:05.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.392+0000 7f1fc193e700 1 -- 172.21.15.43:0/742952408 >> 172.21.15.43:0/742952408 conn(0x7f1fbc0664b0 msgr2=0x7f1fbc0d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:05.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.392+0000 7f1fc193e700 1 -- 172.21.15.43:0/742952408 shutdown_connections 2023-06-28T20:08:05.402 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.392+0000 7f1fc193e700 1 -- 172.21.15.43:0/742952408 wait complete. 2023-06-28T20:08:05.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.393+0000 7f1fc193e700 1 Processor -- start 2023-06-28T20:08:05.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.394+0000 7f1fc193e700 1 -- start start 2023-06-28T20:08:05.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.394+0000 7f1fc193e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fa4003f80 0x7f1fbc15efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:05.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.394+0000 7f1fc193e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fbc151360 0x7f1fbc1f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:05.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.394+0000 7f1fc193e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fbc0b2550 0x7f1fbc0b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:05.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.394+0000 7f1fc193e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc1f4c80 con 0x7f1fa4003f80 2023-06-28T20:08:05.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.394+0000 7f1fc193e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc15aaa0 con 0x7f1fbc151360 2023-06-28T20:08:05.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.394+0000 7f1fc193e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc0b2f00 con 0x7f1fbc0b2550 2023-06-28T20:08:05.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.395+0000 7f1fbaffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fa4003f80 0x7f1fbc15efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:05.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.395+0000 7f1fbaffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fa4003f80 0x7f1fbc15efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54494/0 (socket says 172.21.15.43:54494) 2023-06-28T20:08:05.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.395+0000 7f1fbaffd700 1 -- 172.21.15.43:0/872117970 learned_addr learned my addr 172.21.15.43:0/872117970 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:05.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.395+0000 7f1fbb7fe700 1 --2- 172.21.15.43:0/872117970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fbc0b2550 0x7f1fbc0b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:05.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.395+0000 7f1fba7fc700 1 --2- 172.21.15.43:0/872117970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fbc151360 0x7f1fbc1f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:05.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.395+0000 7f1fbaffd700 1 -- 172.21.15.43:0/872117970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fbc0b2550 msgr2=0x7f1fbc0b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:05.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.395+0000 7f1fbaffd700 1 --2- 172.21.15.43:0/872117970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fbc0b2550 0x7f1fbc0b29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.395+0000 7f1fbaffd700 1 -- 172.21.15.43:0/872117970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fbc151360 msgr2=0x7f1fbc1f4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:05.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.395+0000 7f1fbaffd700 1 --2- 172.21.15.43:0/872117970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fbc151360 0x7f1fbc1f4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.395+0000 7f1fbaffd700 1 -- 172.21.15.43:0/872117970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fbc1588b0 con 0x7f1fa4003f80 2023-06-28T20:08:05.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.395+0000 7f1fbaffd700 1 --2- 172.21.15.43:0/872117970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fa4003f80 0x7f1fbc15efa0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0011a50 tx=0x7f1fb0011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:05.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.395+0000 7f1fa3fff700 1 -- 172.21.15.43:0/872117970 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0005c30 con 0x7f1fa4003f80 2023-06-28T20:08:05.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.395+0000 7f1fa3fff700 1 -- 172.21.15.43:0/872117970 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb0005d90 con 0x7f1fa4003f80 2023-06-28T20:08:05.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.395+0000 7f1fc193e700 1 -- 172.21.15.43:0/872117970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fbc15f6a0 con 0x7f1fa4003f80 2023-06-28T20:08:05.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.395+0000 7f1fa3fff700 1 -- 172.21.15.43:0/872117970 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb000f8f0 con 0x7f1fa4003f80 2023-06-28T20:08:05.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.395+0000 7f1fc193e700 1 -- 172.21.15.43:0/872117970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fbc158b50 con 0x7f1fa4003f80 2023-06-28T20:08:05.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.396+0000 7f1fc193e700 1 -- 172.21.15.43:0/872117970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fbc158ec0 con 0x7f1fa4003f80 2023-06-28T20:08:05.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.398+0000 7f1fa3fff700 1 -- 172.21.15.43:0/872117970 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1fb000fa50 con 0x7f1fa4003f80 2023-06-28T20:08:05.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.398+0000 7f1fa3fff700 1 --2- 172.21.15.43:0/872117970 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1fa806ace0 0x7f1fa806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:05.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.398+0000 7f1fa3fff700 1 -- 172.21.15.43:0/872117970 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1fb0089fe0 con 0x7f1fa4003f80 2023-06-28T20:08:05.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.398+0000 7f1fba7fc700 1 --2- 172.21.15.43:0/872117970 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1fa806ace0 0x7f1fa806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:05.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.398+0000 7f1fba7fc700 1 --2- 172.21.15.43:0/872117970 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1fa806ace0 0x7f1fa806d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1fac000c00 tx=0x7f1fac00a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:05.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.402+0000 7f1fa3fff700 1 -- 172.21.15.43:0/872117970 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1fb00551c0 con 0x7f1fa4003f80 2023-06-28T20:08:05.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.597+0000 7f1fc193e700 1 -- 172.21.15.43:0/872117970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1fbc06a9a0 con 0x7f1fa4003f80 2023-06-28T20:08:05.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.597+0000 7f1fa3fff700 1 -- 172.21.15.43:0/872117970 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1fb00587e0 con 0x7f1fa4003f80 2023-06-28T20:08:05.601 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-06-28T20:08:05.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.599+0000 7f1fc193e700 1 -- 172.21.15.43:0/872117970 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1fa806ace0 msgr2=0x7f1fa806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:05.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.599+0000 7f1fc193e700 1 --2- 172.21.15.43:0/872117970 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1fa806ace0 0x7f1fa806d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1fac000c00 tx=0x7f1fac00a000 comp rx=0 tx=0).stop 2023-06-28T20:08:05.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.599+0000 7f1fc193e700 1 -- 172.21.15.43:0/872117970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fa4003f80 msgr2=0x7f1fbc15efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:05.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.599+0000 7f1fc193e700 1 --2- 172.21.15.43:0/872117970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fa4003f80 0x7f1fbc15efa0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0011a50 tx=0x7f1fb0011d60 comp rx=0 tx=0).stop 2023-06-28T20:08:05.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.599+0000 7f1fc193e700 1 -- 172.21.15.43:0/872117970 shutdown_connections 2023-06-28T20:08:05.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.599+0000 7f1fc193e700 1 --2- 172.21.15.43:0/872117970 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1fa806ace0 0x7f1fa806d190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.599+0000 7f1fc193e700 1 --2- 172.21.15.43:0/872117970 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1fbc0b2550 0x7f1fbc0b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.599+0000 7f1fc193e700 1 --2- 172.21.15.43:0/872117970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fa4003f80 0x7f1fbc15efa0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.599+0000 7f1fc193e700 1 --2- 172.21.15.43:0/872117970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fbc151360 0x7f1fbc1f4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.599+0000 7f1fc193e700 1 -- 172.21.15.43:0/872117970 >> 172.21.15.43:0/872117970 conn(0x7f1fbc0664b0 msgr2=0x7f1fbc0c8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:05.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.599+0000 7f1fc193e700 1 -- 172.21.15.43:0/872117970 shutdown_connections 2023-06-28T20:08:05.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.599+0000 7f1fc193e700 1 -- 172.21.15.43:0/872117970 wait complete. 2023-06-28T20:08:05.615 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-28T20:08:05.743 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.739+0000 7f303a82d700 1 Processor -- start 2023-06-28T20:08:05.744 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.739+0000 7f303a82d700 1 -- start start 2023-06-28T20:08:05.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.740+0000 7f303a82d700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f30341592d0 0x7f30340b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:05.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.740+0000 7f303a82d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3034071900 0x7f30340b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:05.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.740+0000 7f303a82d700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3034070b40 con 0x7f30341596a0 2023-06-28T20:08:05.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.740+0000 7f303a82d700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3034070e20 con 0x7f3034071900 2023-06-28T20:08:05.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.740+0000 7f303a82d700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3034071100 con 0x7f30341592d0 2023-06-28T20:08:05.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.740+0000 7f3038dca700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3034071900 0x7f30340b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:05.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.740+0000 7f3038dca700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3034071900 0x7f30340b1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54502/0 (socket says 172.21.15.43:54502) 2023-06-28T20:08:05.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.740+0000 7f3038dca700 1 -- 172.21.15.43:0/2557523768 learned_addr learned my addr 172.21.15.43:0/2557523768 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:05.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.741+0000 7f3033fff700 1 --2- 172.21.15.43:0/2557523768 >> v2:172.21.15.178:3300/0 conn(0x7f30341592d0 0x7f30340b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:05.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.741+0000 7f3038dca700 1 -- 172.21.15.43:0/2557523768 >> v2:172.21.15.178:3300/0 conn(0x7f30341592d0 msgr2=0x7f30340b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:05.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.741+0000 7f3038dca700 1 --2- 172.21.15.43:0/2557523768 >> v2:172.21.15.178:3300/0 conn(0x7f30341592d0 0x7f30340b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.741+0000 7f3038dca700 1 -- 172.21.15.43:0/2557523768 >> v1:172.21.15.178:6789/0 conn(0x7f30341596a0 legacy=0x7f303415d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:05.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.741+0000 7f3038dca700 1 -- 172.21.15.43:0/2557523768 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30340713e0 con 0x7f3034071900 2023-06-28T20:08:05.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.741+0000 7f3033fff700 1 --2- 172.21.15.43:0/2557523768 >> v2:172.21.15.178:3300/0 conn(0x7f30341592d0 0x7f30340b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:05.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.741+0000 7f3038dca700 1 --2- 172.21.15.43:0/2557523768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3034071900 0x7f30340b1f10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f30340d0bf0 tx=0x7f302c00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876735cf746112ea server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:05.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.741+0000 7f3032ffd700 1 -- 172.21.15.43:0/2557523768 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c010070 con 0x7f3034071900 2023-06-28T20:08:05.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.741+0000 7f3032ffd700 1 -- 172.21.15.43:0/2557523768 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302c00dd90 con 0x7f3034071900 2023-06-28T20:08:05.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.741+0000 7f3032ffd700 1 -- 172.21.15.43:0/2557523768 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c00b310 con 0x7f3034071900 2023-06-28T20:08:05.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.742+0000 7f303a82d700 1 -- 172.21.15.43:0/2557523768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3034071900 msgr2=0x7f30340b1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:05.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.742+0000 7f303a82d700 1 --2- 172.21.15.43:0/2557523768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3034071900 0x7f30340b1f10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f30340d0bf0 tx=0x7f302c00d430 comp rx=0 tx=0).stop 2023-06-28T20:08:05.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.742+0000 7f303a82d700 1 -- 172.21.15.43:0/2557523768 shutdown_connections 2023-06-28T20:08:05.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.742+0000 7f303a82d700 1 --2- 172.21.15.43:0/2557523768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3034071900 0x7f30340b1f10 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.742+0000 7f303a82d700 1 --2- 172.21.15.43:0/2557523768 >> v2:172.21.15.178:3300/0 conn(0x7f30341592d0 0x7f30340b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.742+0000 7f303a82d700 1 -- 172.21.15.43:0/2557523768 >> 172.21.15.43:0/2557523768 conn(0x7f303406b000 msgr2=0x7f303406b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:05.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.742+0000 7f303a82d700 1 -- 172.21.15.43:0/2557523768 shutdown_connections 2023-06-28T20:08:05.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.742+0000 7f303a82d700 1 -- 172.21.15.43:0/2557523768 wait complete. 2023-06-28T20:08:05.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.743+0000 7f303a82d700 1 Processor -- start 2023-06-28T20:08:05.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.743+0000 7f303a82d700 1 -- start start 2023-06-28T20:08:05.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.744+0000 7f303a82d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3034071900 0x7f30340b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:05.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.744+0000 7f303a82d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30341592d0 0x7f30341508a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:05.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.744+0000 7f303a82d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30341596a0 0x7f303414aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:05.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.744+0000 7f303a82d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303415b430 con 0x7f30341596a0 2023-06-28T20:08:05.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.744+0000 7f303a82d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303414f0e0 con 0x7f3034071900 2023-06-28T20:08:05.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.744+0000 7f303a82d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f303414f3c0 con 0x7f30341592d0 2023-06-28T20:08:05.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.744+0000 7f3038dca700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30341596a0 0x7f303414aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:05.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.745+0000 7f3038dca700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30341596a0 0x7f303414aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54510/0 (socket says 172.21.15.43:54510) 2023-06-28T20:08:05.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.745+0000 7f3038dca700 1 -- 172.21.15.43:0/922208661 learned_addr learned my addr 172.21.15.43:0/922208661 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:05.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.745+0000 7f3033fff700 1 --2- 172.21.15.43:0/922208661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3034071900 0x7f30340b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:05.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.745+0000 7f30337fe700 1 --2- 172.21.15.43:0/922208661 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30341592d0 0x7f30341508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:05.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.745+0000 7f3038dca700 1 -- 172.21.15.43:0/922208661 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30341592d0 msgr2=0x7f30341508a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:05.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.745+0000 7f3038dca700 1 --2- 172.21.15.43:0/922208661 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30341592d0 0x7f30341508a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.745+0000 7f3038dca700 1 -- 172.21.15.43:0/922208661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3034071900 msgr2=0x7f30340b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:05.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.745+0000 7f3038dca700 1 --2- 172.21.15.43:0/922208661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3034071900 0x7f30340b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.745+0000 7f3038dca700 1 -- 172.21.15.43:0/922208661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302c00bd40 con 0x7f30341596a0 2023-06-28T20:08:05.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.745+0000 7f3038dca700 1 --2- 172.21.15.43:0/922208661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30341596a0 0x7f303414aba0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f303406bfc0 tx=0x7f302c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:05.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.745+0000 7f3030ff9700 1 -- 172.21.15.43:0/922208661 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c00bbc0 con 0x7f30341596a0 2023-06-28T20:08:05.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.745+0000 7f3030ff9700 1 -- 172.21.15.43:0/922208661 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302c00b4c0 con 0x7f30341596a0 2023-06-28T20:08:05.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.745+0000 7f303a82d700 1 -- 172.21.15.43:0/922208661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303414f6a0 con 0x7f30341596a0 2023-06-28T20:08:05.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.745+0000 7f3030ff9700 1 -- 172.21.15.43:0/922208661 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c00ae70 con 0x7f30341596a0 2023-06-28T20:08:05.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.745+0000 7f303a82d700 1 -- 172.21.15.43:0/922208661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30340c34c0 con 0x7f30341596a0 2023-06-28T20:08:05.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.746+0000 7f303a82d700 1 -- 172.21.15.43:0/922208661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30340cc3e0 con 0x7f30341596a0 2023-06-28T20:08:05.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.747+0000 7f3030ff9700 1 -- 172.21.15.43:0/922208661 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f302c004040 con 0x7f30341596a0 2023-06-28T20:08:05.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.747+0000 7f3030ff9700 1 --2- 172.21.15.43:0/922208661 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f301406ace0 0x7f301406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:05.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.747+0000 7f3030ff9700 1 -- 172.21.15.43:0/922208661 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f302c08cbc0 con 0x7f30341596a0 2023-06-28T20:08:05.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.747+0000 7f3033fff700 1 --2- 172.21.15.43:0/922208661 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f301406ace0 0x7f301406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:05.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.747+0000 7f3033fff700 1 --2- 172.21.15.43:0/922208661 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f301406ace0 0x7f301406d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f30240099d0 tx=0x7f3024008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:05.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.751+0000 7f3030ff9700 1 -- 172.21.15.43:0/922208661 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f302c057d00 con 0x7f30341596a0 2023-06-28T20:08:05.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.949+0000 7f303a82d700 1 -- 172.21.15.43:0/922208661 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f303415b570 con 0x7f30341596a0 2023-06-28T20:08:05.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.950+0000 7f3030ff9700 1 -- 172.21.15.43:0/922208661 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f303415b570 con 0x7f30341596a0 2023-06-28T20:08:05.954 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-06-28T20:08:05.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.952+0000 7f303a82d700 1 -- 172.21.15.43:0/922208661 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f301406ace0 msgr2=0x7f301406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:05.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.952+0000 7f303a82d700 1 --2- 172.21.15.43:0/922208661 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f301406ace0 0x7f301406d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f30240099d0 tx=0x7f3024008040 comp rx=0 tx=0).stop 2023-06-28T20:08:05.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.952+0000 7f303a82d700 1 -- 172.21.15.43:0/922208661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30341596a0 msgr2=0x7f303414aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:05.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.952+0000 7f303a82d700 1 --2- 172.21.15.43:0/922208661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30341596a0 0x7f303414aba0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f303406bfc0 tx=0x7f302c004ad0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.956 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.952+0000 7f303a82d700 1 -- 172.21.15.43:0/922208661 shutdown_connections 2023-06-28T20:08:05.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.952+0000 7f303a82d700 1 --2- 172.21.15.43:0/922208661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30341596a0 0x7f303414aba0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.952+0000 7f303a82d700 1 --2- 172.21.15.43:0/922208661 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3034071900 0x7f30340b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.952+0000 7f303a82d700 1 --2- 172.21.15.43:0/922208661 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f301406ace0 0x7f301406d190 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.952+0000 7f303a82d700 1 --2- 172.21.15.43:0/922208661 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f30341592d0 0x7f30341508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:05.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.952+0000 7f303a82d700 1 -- 172.21.15.43:0/922208661 >> 172.21.15.43:0/922208661 conn(0x7f303406b000 msgr2=0x7f3034154e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:05.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.952+0000 7f303a82d700 1 -- 172.21.15.43:0/922208661 shutdown_connections 2023-06-28T20:08:05.957 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:05.952+0000 7f303a82d700 1 -- 172.21.15.43:0/922208661 wait complete. 2023-06-28T20:08:05.968 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-28T20:08:06.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.091+0000 7fb5b06b4700 1 Processor -- start 2023-06-28T20:08:06.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.091+0000 7fb5b06b4700 1 -- start start 2023-06-28T20:08:06.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.092+0000 7fb5b06b4700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fb5a806c0a0 0x7fb5a806c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:06.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.092+0000 7fb5b06b4700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5a815cce0 con 0x7fb5a80cfe20 2023-06-28T20:08:06.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.092+0000 7fb5b06b4700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5a806c9e0 con 0x7fb5a815a180 2023-06-28T20:08:06.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.092+0000 7fb5b06b4700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a80ae770 con 0x7fb5a806c0a0 2023-06-28T20:08:06.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.093+0000 7fb5adc4f700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb5a80cfe20 0x7fb5a815d450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43700/0 (socket says 172.21.15.43:43700) 2023-06-28T20:08:06.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.093+0000 7fb5adc4f700 1 -- 172.21.15.43:0/1334694985 learned_addr learned my addr 172.21.15.43:0/1334694985 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:06.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.093+0000 7fb5aec51700 1 --2- 172.21.15.43:0/1334694985 >> v2:172.21.15.178:3300/0 conn(0x7fb5a806c0a0 0x7fb5a806c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:06.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.093+0000 7fb5ad44e700 1 -- 172.21.15.43:0/1334694985 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541384427 0 0) 0x7fb5a815cce0 con 0x7fb5a80cfe20 2023-06-28T20:08:06.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.093+0000 7fb5ad44e700 1 -- 172.21.15.43:0/1334694985 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5940035b0 con 0x7fb5a80cfe20 2023-06-28T20:08:06.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.093+0000 7fb5aec51700 1 -- 172.21.15.43:0/1334694985 >> v1:172.21.15.178:6789/0 conn(0x7fb5a815a180 legacy=0x7fb5a80ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:06.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.093+0000 7fb5aec51700 1 -- 172.21.15.43:0/1334694985 >> v1:172.21.15.43:6789/0 conn(0x7fb5a80cfe20 legacy=0x7fb5a815d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:06.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.093+0000 7fb5aec51700 1 -- 172.21.15.43:0/1334694985 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a815db60 con 0x7fb5a806c0a0 2023-06-28T20:08:06.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.093+0000 7fb5aec51700 1 --2- 172.21.15.43:0/1334694985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5a806c0a0 0x7fb5a806c4a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0009d90 tx=0x7fb5a000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0cdb3ccfe7b3684 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:06.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.096+0000 7fb5ad44e700 1 -- 172.21.15.43:0/1334694985 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a000e070 con 0x7fb5a806c0a0 2023-06-28T20:08:06.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.096+0000 7fb5ad44e700 1 -- 172.21.15.43:0/1334694985 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a000bb20 con 0x7fb5a806c0a0 2023-06-28T20:08:06.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.096+0000 7fb5ad44e700 1 -- 172.21.15.43:0/1334694985 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0012430 con 0x7fb5a806c0a0 2023-06-28T20:08:06.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.097+0000 7fb5b06b4700 1 -- 172.21.15.43:0/1334694985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5a806c0a0 msgr2=0x7fb5a806c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.097+0000 7fb5b06b4700 1 --2- 172.21.15.43:0/1334694985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5a806c0a0 0x7fb5a806c4a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0009d90 tx=0x7fb5a000b330 comp rx=0 tx=0).stop 2023-06-28T20:08:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.097+0000 7fb5b06b4700 1 -- 172.21.15.43:0/1334694985 shutdown_connections 2023-06-28T20:08:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.097+0000 7fb5b06b4700 1 --2- 172.21.15.43:0/1334694985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5a806c0a0 0x7fb5a806c4a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.097+0000 7fb5b06b4700 1 -- 172.21.15.43:0/1334694985 >> 172.21.15.43:0/1334694985 conn(0x7fb5a80b1a50 msgr2=0x7fb5a80cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.097+0000 7fb5b06b4700 1 -- 172.21.15.43:0/1334694985 shutdown_connections 2023-06-28T20:08:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.097+0000 7fb5b06b4700 1 -- 172.21.15.43:0/1334694985 wait complete. 2023-06-28T20:08:06.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.098+0000 7fb5b06b4700 1 Processor -- start 2023-06-28T20:08:06.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.098+0000 7fb5b06b4700 1 -- start start 2023-06-28T20:08:06.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.099+0000 7fb5b06b4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5a806c0a0 0x7fb5a80c3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:06.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.099+0000 7fb5b06b4700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5a80cfe20 0x7fb5a81f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:06.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.099+0000 7fb5b06b4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5a815a180 0x7fb5a806a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:06.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.099+0000 7fb5b06b4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a80c3640 con 0x7fb5a806c0a0 2023-06-28T20:08:06.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.099+0000 7fb5b06b4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a81f5010 con 0x7fb5a815a180 2023-06-28T20:08:06.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.099+0000 7fb5b06b4700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a80c4f30 con 0x7fb5a80cfe20 2023-06-28T20:08:06.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.099+0000 7fb5ae450700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5a806c0a0 0x7fb5a80c3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:06.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.100+0000 7fb5ae450700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5a806c0a0 0x7fb5a80c3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54518/0 (socket says 172.21.15.43:54518) 2023-06-28T20:08:06.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.100+0000 7fb5ae450700 1 -- 172.21.15.43:0/3472409812 learned_addr learned my addr 172.21.15.43:0/3472409812 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:06.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.100+0000 7fb5aec51700 1 --2- 172.21.15.43:0/3472409812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5a815a180 0x7fb5a806a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:06.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.100+0000 7fb5adc4f700 1 --2- 172.21.15.43:0/3472409812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5a80cfe20 0x7fb5a81f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:06.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.100+0000 7fb5ae450700 1 -- 172.21.15.43:0/3472409812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5a80cfe20 msgr2=0x7fb5a81f4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:06.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.100+0000 7fb5ae450700 1 --2- 172.21.15.43:0/3472409812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5a80cfe20 0x7fb5a81f4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:06.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.100+0000 7fb5ae450700 1 -- 172.21.15.43:0/3472409812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5a815a180 msgr2=0x7fb5a806a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:06.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.100+0000 7fb5ae450700 1 --2- 172.21.15.43:0/3472409812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5a815a180 0x7fb5a806a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:06.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.100+0000 7fb5ae450700 1 -- 172.21.15.43:0/3472409812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a0009a70 con 0x7fb5a806c0a0 2023-06-28T20:08:06.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.100+0000 7fb5ae450700 1 --2- 172.21.15.43:0/3472409812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5a806c0a0 0x7fb5a80c3100 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400c4e0 tx=0x7fb5a400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:06.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.100+0000 7fb59effd700 1 -- 172.21.15.43:0/3472409812 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a40052d0 con 0x7fb5a806c0a0 2023-06-28T20:08:06.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.100+0000 7fb59effd700 1 -- 172.21.15.43:0/3472409812 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a4005430 con 0x7fb5a806c0a0 2023-06-28T20:08:06.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.100+0000 7fb59effd700 1 -- 172.21.15.43:0/3472409812 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a40038c0 con 0x7fb5a806c0a0 2023-06-28T20:08:06.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.100+0000 7fb5b06b4700 1 -- 172.21.15.43:0/3472409812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a80c5210 con 0x7fb5a806c0a0 2023-06-28T20:08:06.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.100+0000 7fb5b06b4700 1 -- 172.21.15.43:0/3472409812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a80c57d0 con 0x7fb5a806c0a0 2023-06-28T20:08:06.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.101+0000 7fb5b06b4700 1 -- 172.21.15.43:0/3472409812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5a80728b0 con 0x7fb5a806c0a0 2023-06-28T20:08:06.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.102+0000 7fb59effd700 1 -- 172.21.15.43:0/3472409812 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb5a4003a20 con 0x7fb5a806c0a0 2023-06-28T20:08:06.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.102+0000 7fb59effd700 1 --2- 172.21.15.43:0/3472409812 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb58c06ace0 0x7fb58c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:06.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.102+0000 7fb5adc4f700 1 --2- 172.21.15.43:0/3472409812 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb58c06ace0 0x7fb58c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:06.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.102+0000 7fb59effd700 1 -- 172.21.15.43:0/3472409812 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb5a4088820 con 0x7fb5a806c0a0 2023-06-28T20:08:06.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.103+0000 7fb5adc4f700 1 --2- 172.21.15.43:0/3472409812 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb58c06ace0 0x7fb58c06d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb5a80c2b80 tx=0x7fb598007dc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:06.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.107+0000 7fb59effd700 1 -- 172.21.15.43:0/3472409812 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb5a40538d0 con 0x7fb5a806c0a0 2023-06-28T20:08:06.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.305+0000 7fb5b06b4700 1 -- 172.21.15.43:0/3472409812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb5a80cfc00 con 0x7fb5a806c0a0 2023-06-28T20:08:06.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.305+0000 7fb59effd700 1 -- 172.21.15.43:0/3472409812 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb5a4017020 con 0x7fb5a806c0a0 2023-06-28T20:08:06.309 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-06-28T20:08:06.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.307+0000 7fb5b06b4700 1 -- 172.21.15.43:0/3472409812 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb58c06ace0 msgr2=0x7fb58c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.307+0000 7fb5b06b4700 1 --2- 172.21.15.43:0/3472409812 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb58c06ace0 0x7fb58c06d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb5a80c2b80 tx=0x7fb598007dc0 comp rx=0 tx=0).stop 2023-06-28T20:08:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.307+0000 7fb5b06b4700 1 -- 172.21.15.43:0/3472409812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5a806c0a0 msgr2=0x7fb5a80c3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.307+0000 7fb5b06b4700 1 --2- 172.21.15.43:0/3472409812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5a806c0a0 0x7fb5a80c3100 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400c4e0 tx=0x7fb5a400c7f0 comp rx=0 tx=0).stop 2023-06-28T20:08:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.307+0000 7fb5b06b4700 1 -- 172.21.15.43:0/3472409812 shutdown_connections 2023-06-28T20:08:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.307+0000 7fb5b06b4700 1 --2- 172.21.15.43:0/3472409812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5a815a180 0x7fb5a806a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.307+0000 7fb5b06b4700 1 --2- 172.21.15.43:0/3472409812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5a806c0a0 0x7fb5a80c3100 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:06.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.307+0000 7fb5b06b4700 1 --2- 172.21.15.43:0/3472409812 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb58c06ace0 0x7fb58c06d190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:06.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.307+0000 7fb5b06b4700 1 --2- 172.21.15.43:0/3472409812 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb5a80cfe20 0x7fb5a81f4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:06.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.307+0000 7fb5b06b4700 1 -- 172.21.15.43:0/3472409812 >> 172.21.15.43:0/3472409812 conn(0x7fb5a80b1a50 msgr2=0x7fb5a80b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:06.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.307+0000 7fb5b06b4700 1 -- 172.21.15.43:0/3472409812 shutdown_connections 2023-06-28T20:08:06.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.308+0000 7fb5b06b4700 1 -- 172.21.15.43:0/3472409812 wait complete. 2023-06-28T20:08:06.323 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-28T20:08:06.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.445+0000 7fbed19e3700 1 Processor -- start 2023-06-28T20:08:06.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.445+0000 7fbed19e3700 1 -- start start 2023-06-28T20:08:06.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.445+0000 7fbed19e3700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbecc0ce170 0x7fbecc0c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:06.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.445+0000 7fbed19e3700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbecc15a960 con 0x7fbecc14ff80 2023-06-28T20:08:06.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.445+0000 7fbed19e3700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbecc0ca490 con 0x7fbecc0ce870 2023-06-28T20:08:06.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.445+0000 7fbed19e3700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc0ca770 con 0x7fbecc0ce170 2023-06-28T20:08:06.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.446+0000 7fbecaffd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbecc14ff80 0x7fbecc150350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43712/0 (socket says 172.21.15.43:43712) 2023-06-28T20:08:06.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.446+0000 7fbecaffd700 1 -- 172.21.15.43:0/2412602589 learned_addr learned my addr 172.21.15.43:0/2412602589 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:06.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.446+0000 7fbeca7fc700 1 --2- 172.21.15.43:0/2412602589 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbecc0ce170 0x7fbecc0c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:06.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.446+0000 7fbec9ffb700 1 -- 172.21.15.43:0/2412602589 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 632931952 0 0) 0x7fbecc15a960 con 0x7fbecc14ff80 2023-06-28T20:08:06.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.446+0000 7fbec9ffb700 1 -- 172.21.15.43:0/2412602589 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbeb40035b0 con 0x7fbecc14ff80 2023-06-28T20:08:06.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.446+0000 7fbec9ffb700 1 -- 172.21.15.43:0/2412602589 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936737733 0 0) 0x7fbecc0ca490 con 0x7fbecc0ce870 2023-06-28T20:08:06.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.446+0000 7fbec9ffb700 1 -- 172.21.15.43:0/2412602589 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbecc15a960 con 0x7fbecc0ce870 2023-06-28T20:08:06.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.446+0000 7fbeca7fc700 1 -- 172.21.15.43:0/2412602589 >> v1:172.21.15.178:6789/0 conn(0x7fbecc0ce870 legacy=0x7fbecc069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:06.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.446+0000 7fbeca7fc700 1 -- 172.21.15.43:0/2412602589 >> v1:172.21.15.43:6789/0 conn(0x7fbecc14ff80 legacy=0x7fbecc150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:06.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.446+0000 7fbec9ffb700 1 -- 172.21.15.43:0/2412602589 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1260802537 0 0) 0x7fbeb40035b0 con 0x7fbecc14ff80 2023-06-28T20:08:06.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.446+0000 7fbeca7fc700 1 -- 172.21.15.43:0/2412602589 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc0cd6b0 con 0x7fbecc0ce170 2023-06-28T20:08:06.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.446+0000 7fbeca7fc700 1 --2- 172.21.15.43:0/2412602589 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbecc0ce170 0x7fbecc0c9c20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009870 tx=0x7fbec0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e57d42a96f77156 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:06.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.447+0000 7fbec9ffb700 1 -- 172.21.15.43:0/2412602589 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc001670 con 0x7fbecc0ce170 2023-06-28T20:08:06.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.447+0000 7fbec9ffb700 1 -- 172.21.15.43:0/2412602589 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec0004030 con 0x7fbecc0ce170 2023-06-28T20:08:06.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.447+0000 7fbec9ffb700 1 -- 172.21.15.43:0/2412602589 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec00043c0 con 0x7fbecc0ce170 2023-06-28T20:08:06.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.447+0000 7fbed19e3700 1 -- 172.21.15.43:0/2412602589 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbecc0ce170 msgr2=0x7fbecc0c9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:06.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.447+0000 7fbed19e3700 1 --2- 172.21.15.43:0/2412602589 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbecc0ce170 0x7fbecc0c9c20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009870 tx=0x7fbec0009b80 comp rx=0 tx=0).stop 2023-06-28T20:08:06.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.447+0000 7fbed19e3700 1 -- 172.21.15.43:0/2412602589 shutdown_connections 2023-06-28T20:08:06.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.447+0000 7fbed19e3700 1 --2- 172.21.15.43:0/2412602589 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbecc0ce170 0x7fbecc0c9c20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:06.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.447+0000 7fbed19e3700 1 -- 172.21.15.43:0/2412602589 >> 172.21.15.43:0/2412602589 conn(0x7fbecc06c170 msgr2=0x7fbecc06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:06.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.447+0000 7fbed19e3700 1 -- 172.21.15.43:0/2412602589 shutdown_connections 2023-06-28T20:08:06.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.447+0000 7fbed19e3700 1 -- 172.21.15.43:0/2412602589 wait complete. 2023-06-28T20:08:06.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.448+0000 7fbed19e3700 1 Processor -- start 2023-06-28T20:08:06.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.449+0000 7fbed19e3700 1 -- start start 2023-06-28T20:08:06.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.449+0000 7fbed19e3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbecc0ce170 0x7fbecc1588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:06.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.449+0000 7fbed19e3700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbecc0ce870 0x7fbecc14f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:06.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.449+0000 7fbed19e3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbecc14ff80 0x7fbecc1f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:06.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.449+0000 7fbed19e3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc1f2080 con 0x7fbecc0ce170 2023-06-28T20:08:06.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.449+0000 7fbed19e3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc1f1d00 con 0x7fbecc14ff80 2023-06-28T20:08:06.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.449+0000 7fbed19e3700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc1f5830 con 0x7fbecc0ce870 2023-06-28T20:08:06.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.449+0000 7fbecaffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbecc0ce170 0x7fbecc1588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:06.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.449+0000 7fbecaffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbecc0ce170 0x7fbecc1588c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54532/0 (socket says 172.21.15.43:54532) 2023-06-28T20:08:06.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.449+0000 7fbecaffd700 1 -- 172.21.15.43:0/2172054767 learned_addr learned my addr 172.21.15.43:0/2172054767 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:06.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.450+0000 7fbeca7fc700 1 --2- 172.21.15.43:0/2172054767 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbecc0ce870 0x7fbecc14f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:06.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.450+0000 7fbecb7fe700 1 --2- 172.21.15.43:0/2172054767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbecc14ff80 0x7fbecc1f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:06.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.450+0000 7fbecaffd700 1 -- 172.21.15.43:0/2172054767 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbecc0ce870 msgr2=0x7fbecc14f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:06.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.450+0000 7fbecaffd700 1 --2- 172.21.15.43:0/2172054767 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbecc0ce870 0x7fbecc14f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:06.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.450+0000 7fbecaffd700 1 -- 172.21.15.43:0/2172054767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbecc14ff80 msgr2=0x7fbecc1f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:06.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.450+0000 7fbecaffd700 1 --2- 172.21.15.43:0/2172054767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbecc14ff80 0x7fbecc1f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:06.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.450+0000 7fbecaffd700 1 -- 172.21.15.43:0/2172054767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec00094a0 con 0x7fbecc0ce170 2023-06-28T20:08:06.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.450+0000 7fbecaffd700 1 --2- 172.21.15.43:0/2172054767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbecc0ce170 0x7fbecc1588c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00c6b0 tx=0x7fbebc00ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:06.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.450+0000 7fbeb3fff700 1 -- 172.21.15.43:0/2172054767 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc004c80 con 0x7fbecc0ce170 2023-06-28T20:08:06.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.450+0000 7fbeb3fff700 1 -- 172.21.15.43:0/2172054767 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbebc004e00 con 0x7fbecc0ce170 2023-06-28T20:08:06.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.450+0000 7fbed19e3700 1 -- 172.21.15.43:0/2172054767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbecc1f5ad0 con 0x7fbecc0ce170 2023-06-28T20:08:06.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.450+0000 7fbeb3fff700 1 -- 172.21.15.43:0/2172054767 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc007310 con 0x7fbecc0ce170 2023-06-28T20:08:06.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.450+0000 7fbed19e3700 1 -- 172.21.15.43:0/2172054767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbecc1f6090 con 0x7fbecc0ce170 2023-06-28T20:08:06.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.451+0000 7fbed19e3700 1 -- 172.21.15.43:0/2172054767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbecc1f6280 con 0x7fbecc0ce170 2023-06-28T20:08:06.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.452+0000 7fbeb3fff700 1 -- 172.21.15.43:0/2172054767 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbebc007470 con 0x7fbecc0ce170 2023-06-28T20:08:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.452+0000 7fbeb3fff700 1 --2- 172.21.15.43:0/2172054767 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbeb806ace0 0x7fbeb806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.452+0000 7fbeca7fc700 1 --2- 172.21.15.43:0/2172054767 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbeb806ace0 0x7fbeb806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.452+0000 7fbeb3fff700 1 -- 172.21.15.43:0/2172054767 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbebc089230 con 0x7fbecc0ce170 2023-06-28T20:08:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.453+0000 7fbeca7fc700 1 --2- 172.21.15.43:0/2172054767 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbeb806ace0 0x7fbeb806d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbec00097c0 tx=0x7fbec0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.457+0000 7fbeb3fff700 1 -- 172.21.15.43:0/2172054767 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbecc1f6280 con 0x7fbecc0ce170 2023-06-28T20:08:06.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.699+0000 7fbed19e3700 1 -- 172.21.15.43:0/2172054767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbecc1f6280 con 0x7fbecc0ce170 2023-06-28T20:08:06.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.700+0000 7fbeb3fff700 1 -- 172.21.15.43:0/2172054767 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4665 (secure 0 0 0) 0x7fbecc1f6280 con 0x7fbecc0ce170 2023-06-28T20:08:06.705 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:06.705 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4033115951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4033115951},{"type":"v1","addr":"172.21.15.178:6835","nonce":4033115951}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/65569869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":65569869},{"type":"v1","addr":"172.21.15.43:6837","nonce":65569869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/1016213057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016213057},{"type":"v1","addr":"172.21.15.178:6834","nonce":1016213057}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-28T20:07:20.788417+0000","modified":"2023-06-28T20:07:24.388714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/3514544484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3514544484},{"type":"v1","addr":"172.21.15.43:6835","nonce":3514544484}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-28T20:08:06.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.702+0000 7fbed19e3700 1 -- 172.21.15.43:0/2172054767 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbeb806ace0 msgr2=0x7fbeb806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:06.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.702+0000 7fbed19e3700 1 --2- 172.21.15.43:0/2172054767 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbeb806ace0 0x7fbeb806d190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbec00097c0 tx=0x7fbec0008030 comp rx=0 tx=0).stop 2023-06-28T20:08:06.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.702+0000 7fbed19e3700 1 -- 172.21.15.43:0/2172054767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbecc0ce170 msgr2=0x7fbecc1588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:06.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.702+0000 7fbed19e3700 1 --2- 172.21.15.43:0/2172054767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbecc0ce170 0x7fbecc1588c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00c6b0 tx=0x7fbebc00ca70 comp rx=0 tx=0).stop 2023-06-28T20:08:06.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.702+0000 7fbed19e3700 1 -- 172.21.15.43:0/2172054767 shutdown_connections 2023-06-28T20:08:06.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.702+0000 7fbed19e3700 1 --2- 172.21.15.43:0/2172054767 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbecc14ff80 0x7fbecc1f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:06.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.702+0000 7fbed19e3700 1 --2- 172.21.15.43:0/2172054767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbecc0ce170 0x7fbecc1588c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:06.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.702+0000 7fbed19e3700 1 --2- 172.21.15.43:0/2172054767 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbeb806ace0 0x7fbeb806d190 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:06.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.702+0000 7fbed19e3700 1 --2- 172.21.15.43:0/2172054767 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbecc0ce870 0x7fbecc14f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:06.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.702+0000 7fbed19e3700 1 -- 172.21.15.43:0/2172054767 >> 172.21.15.43:0/2172054767 conn(0x7fbecc06c170 msgr2=0x7fbecc15bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:06.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.702+0000 7fbed19e3700 1 -- 172.21.15.43:0/2172054767 shutdown_connections 2023-06-28T20:08:06.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.702+0000 7fbed19e3700 1 -- 172.21.15.43:0/2172054767 wait complete. 2023-06-28T20:08:06.708 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 7 2023-06-28T20:08:06.718 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:08:06.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.838+0000 7fbe3fa0d700 1 Processor -- start 2023-06-28T20:08:06.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.839+0000 7fbe3fa0d700 1 -- start start 2023-06-28T20:08:06.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.839+0000 7fbe3fa0d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe380d10f0 0x7fbe380ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:06.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.839+0000 7fbe3fa0d700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbe380cedf0 0x7fbe380cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:06.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.839+0000 7fbe3fa0d700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3815ab50 con 0x7fbe380cb8e0 2023-06-28T20:08:06.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.839+0000 7fbe3fa0d700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe380cf260 con 0x7fbe380cedf0 2023-06-28T20:08:06.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.839+0000 7fbe3fa0d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe380cd1b0 con 0x7fbe380d10f0 2023-06-28T20:08:06.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.840+0000 7fbe3dfaa700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbe380cb8e0 0x7fbe380cd600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43728/0 (socket says 172.21.15.43:43728) 2023-06-28T20:08:06.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.840+0000 7fbe3dfaa700 1 -- 172.21.15.43:0/4229974705 learned_addr learned my addr 172.21.15.43:0/4229974705 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:06.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.840+0000 7fbe3cfa8700 1 --2- 172.21.15.43:0/4229974705 >> v2:172.21.15.178:3300/0 conn(0x7fbe380cedf0 0x7fbe380cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:06.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.840+0000 7fbe3d7a9700 1 --2- 172.21.15.43:0/4229974705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe380d10f0 0x7fbe380ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:06.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.840+0000 7fbe2ffff700 1 -- 172.21.15.43:0/4229974705 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1602554607 0 0) 0x7fbe3815ab50 con 0x7fbe380cb8e0 2023-06-28T20:08:06.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.840+0000 7fbe2ffff700 1 -- 172.21.15.43:0/4229974705 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe240035b0 con 0x7fbe380cb8e0 2023-06-28T20:08:06.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.840+0000 7fbe3cfa8700 1 -- 172.21.15.43:0/4229974705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe380d10f0 msgr2=0x7fbe380ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:06.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.840+0000 7fbe2ffff700 1 -- 172.21.15.43:0/4229974705 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3290884868 0 0) 0x7fbe240035b0 con 0x7fbe380cb8e0 2023-06-28T20:08:06.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.840+0000 7fbe3cfa8700 1 --2- 172.21.15.43:0/4229974705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe380d10f0 0x7fbe380ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:06.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.840+0000 7fbe3cfa8700 1 -- 172.21.15.43:0/4229974705 >> v1:172.21.15.43:6789/0 conn(0x7fbe380cb8e0 legacy=0x7fbe380cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:06.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.840+0000 7fbe3cfa8700 1 -- 172.21.15.43:0/4229974705 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe380cc110 con 0x7fbe380cedf0 2023-06-28T20:08:06.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.840+0000 7fbe3cfa8700 1 --2- 172.21.15.43:0/4229974705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe380cedf0 0x7fbe380cb3a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009840 tx=0x7fbe34009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d068890abc5b62cc server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:06.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.841+0000 7fbe2ffff700 1 -- 172.21.15.43:0/4229974705 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe300019f0 con 0x7fbe380cedf0 2023-06-28T20:08:06.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.841+0000 7fbe2ffff700 1 -- 172.21.15.43:0/4229974705 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe34004030 con 0x7fbe380cedf0 2023-06-28T20:08:06.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.841+0000 7fbe2ffff700 1 -- 172.21.15.43:0/4229974705 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe340043c0 con 0x7fbe380cedf0 2023-06-28T20:08:06.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.841+0000 7fbe3fa0d700 1 -- 172.21.15.43:0/4229974705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe380cedf0 msgr2=0x7fbe380cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:06.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.841+0000 7fbe3fa0d700 1 --2- 172.21.15.43:0/4229974705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe380cedf0 0x7fbe380cb3a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009840 tx=0x7fbe34009b50 comp rx=0 tx=0).stop 2023-06-28T20:08:06.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.841+0000 7fbe3fa0d700 1 -- 172.21.15.43:0/4229974705 shutdown_connections 2023-06-28T20:08:06.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.841+0000 7fbe3fa0d700 1 --2- 172.21.15.43:0/4229974705 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe380d10f0 0x7fbe380ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:06.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.841+0000 7fbe3fa0d700 1 --2- 172.21.15.43:0/4229974705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe380cedf0 0x7fbe380cb3a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:06.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.841+0000 7fbe3fa0d700 1 -- 172.21.15.43:0/4229974705 >> 172.21.15.43:0/4229974705 conn(0x7fbe380664b0 msgr2=0x7fbe3809a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:06.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.841+0000 7fbe3fa0d700 1 -- 172.21.15.43:0/4229974705 shutdown_connections 2023-06-28T20:08:06.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.841+0000 7fbe3fa0d700 1 -- 172.21.15.43:0/4229974705 wait complete. 2023-06-28T20:08:06.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.843+0000 7fbe3fa0d700 1 Processor -- start 2023-06-28T20:08:06.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.843+0000 7fbe3fa0d700 1 -- start start 2023-06-28T20:08:06.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.844+0000 7fbe3fa0d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe380cb8e0 0x7fbe3814b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:06.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.844+0000 7fbe3fa0d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe380cedf0 0x7fbe3815bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.844+0000 7fbe3fa0d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe380d10f0 0x7fbe380b6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.844+0000 7fbe3fa0d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3815d1c0 con 0x7fbe380cb8e0 2023-06-28T20:08:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.844+0000 7fbe3fa0d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3815c270 con 0x7fbe380d10f0 2023-06-28T20:08:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.844+0000 7fbe3fa0d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3815c540 con 0x7fbe380cedf0 2023-06-28T20:08:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.844+0000 7fbe3d7a9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe380cb8e0 0x7fbe3814b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.844+0000 7fbe3d7a9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe380cb8e0 0x7fbe3814b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54544/0 (socket says 172.21.15.43:54544) 2023-06-28T20:08:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.844+0000 7fbe3d7a9700 1 -- 172.21.15.43:0/3300053335 learned_addr learned my addr 172.21.15.43:0/3300053335 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.844+0000 7fbe3dfaa700 1 --2- 172.21.15.43:0/3300053335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe380d10f0 0x7fbe380b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.844+0000 7fbe3cfa8700 1 --2- 172.21.15.43:0/3300053335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe380cedf0 0x7fbe3815bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.844+0000 7fbe3d7a9700 1 -- 172.21.15.43:0/3300053335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe380cedf0 msgr2=0x7fbe3815bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.844+0000 7fbe3d7a9700 1 --2- 172.21.15.43:0/3300053335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe380cedf0 0x7fbe3815bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.844+0000 7fbe3d7a9700 1 -- 172.21.15.43:0/3300053335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe380d10f0 msgr2=0x7fbe380b6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:06.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.844+0000 7fbe3d7a9700 1 --2- 172.21.15.43:0/3300053335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe380d10f0 0x7fbe380b6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:06.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.844+0000 7fbe3d7a9700 1 -- 172.21.15.43:0/3300053335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe340094a0 con 0x7fbe380cb8e0 2023-06-28T20:08:06.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.844+0000 7fbe3dfaa700 1 --2- 172.21.15.43:0/3300053335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe380d10f0 0x7fbe380b6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:06.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.844+0000 7fbe3cfa8700 1 --2- 172.21.15.43:0/3300053335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe380cedf0 0x7fbe3815bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:06.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.844+0000 7fbe3d7a9700 1 --2- 172.21.15.43:0/3300053335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe380cb8e0 0x7fbe3814b0c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800e900 tx=0x7fbe2800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:06.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.844+0000 7fbe2dffb700 1 -- 172.21.15.43:0/3300053335 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28009650 con 0x7fbe380cb8e0 2023-06-28T20:08:06.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.845+0000 7fbe2dffb700 1 -- 172.21.15.43:0/3300053335 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2800c850 con 0x7fbe380cb8e0 2023-06-28T20:08:06.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.845+0000 7fbe3fa0d700 1 -- 172.21.15.43:0/3300053335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3815c820 con 0x7fbe380cb8e0 2023-06-28T20:08:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.845+0000 7fbe2dffb700 1 -- 172.21.15.43:0/3300053335 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2800cbc0 con 0x7fbe380cb8e0 2023-06-28T20:08:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.845+0000 7fbe3fa0d700 1 -- 172.21.15.43:0/3300053335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3815cde0 con 0x7fbe380cb8e0 2023-06-28T20:08:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.846+0000 7fbe3fa0d700 1 -- 172.21.15.43:0/3300053335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3815d430 con 0x7fbe380cb8e0 2023-06-28T20:08:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.846+0000 7fbe2dffb700 1 -- 172.21.15.43:0/3300053335 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbe2800cd20 con 0x7fbe380cb8e0 2023-06-28T20:08:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.847+0000 7fbe2dffb700 1 --2- 172.21.15.43:0/3300053335 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbe1c06ace0 0x7fbe1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.847+0000 7fbe2dffb700 1 -- 172.21.15.43:0/3300053335 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbe280899a0 con 0x7fbe380cb8e0 2023-06-28T20:08:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.847+0000 7fbe3cfa8700 1 --2- 172.21.15.43:0/3300053335 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbe1c06ace0 0x7fbe1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.847+0000 7fbe3cfa8700 1 --2- 172.21.15.43:0/3300053335 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbe1c06ace0 0x7fbe1c06d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbe381f38c0 tx=0x7fbe34008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:06.851+0000 7fbe2dffb700 1 -- 172.21.15.43:0/3300053335 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbe280549b0 con 0x7fbe380cb8e0 2023-06-28T20:08:07.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.053+0000 7fbe3fa0d700 1 -- 172.21.15.43:0/3300053335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe380cfda0 con 0x7fbe380cb8e0 2023-06-28T20:08:07.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.054+0000 7fbe2dffb700 1 -- 172.21.15.43:0/3300053335 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+18221 (secure 0 0 0) 0x7fbe28017090 con 0x7fbe380cb8e0 2023-06-28T20:08:07.059 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:07.059 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:07:22.377248+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:07:18.624759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-28T20:07:19.760017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:08:07.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.056+0000 7fbe3fa0d700 1 -- 172.21.15.43:0/3300053335 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbe1c06ace0 msgr2=0x7fbe1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:07.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.056+0000 7fbe3fa0d700 1 --2- 172.21.15.43:0/3300053335 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbe1c06ace0 0x7fbe1c06d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbe381f38c0 tx=0x7fbe34008030 comp rx=0 tx=0).stop 2023-06-28T20:08:07.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.056+0000 7fbe3fa0d700 1 -- 172.21.15.43:0/3300053335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe380cb8e0 msgr2=0x7fbe3814b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:07.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.056+0000 7fbe3fa0d700 1 --2- 172.21.15.43:0/3300053335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe380cb8e0 0x7fbe3814b0c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800e900 tx=0x7fbe2800ecc0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.056+0000 7fbe3fa0d700 1 -- 172.21.15.43:0/3300053335 shutdown_connections 2023-06-28T20:08:07.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.056+0000 7fbe3fa0d700 1 --2- 172.21.15.43:0/3300053335 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbe380d10f0 0x7fbe380b6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.056+0000 7fbe3fa0d700 1 --2- 172.21.15.43:0/3300053335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe380cb8e0 0x7fbe3814b0c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.056+0000 7fbe3fa0d700 1 --2- 172.21.15.43:0/3300053335 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbe1c06ace0 0x7fbe1c06d190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.056+0000 7fbe3fa0d700 1 --2- 172.21.15.43:0/3300053335 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbe380cedf0 0x7fbe3815bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.056+0000 7fbe3fa0d700 1 -- 172.21.15.43:0/3300053335 >> 172.21.15.43:0/3300053335 conn(0x7fbe380664b0 msgr2=0x7fbe380d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:07.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.056+0000 7fbe3fa0d700 1 -- 172.21.15.43:0/3300053335 shutdown_connections 2023-06-28T20:08:07.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.056+0000 7fbe3fa0d700 1 -- 172.21.15.43:0/3300053335 wait complete. 2023-06-28T20:08:07.073 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-06-28T20:08:07.073 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-28T20:08:07.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.194+0000 7fd11c454700 1 Processor -- start 2023-06-28T20:08:07.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.194+0000 7fd11c454700 1 -- start start 2023-06-28T20:08:07.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.195+0000 7fd11c454700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd1140d10f0 0x7fd1140ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:07.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.195+0000 7fd11c454700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd1140cedf0 0x7fd1140cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:07.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.195+0000 7fd11c454700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd11415ab50 con 0x7fd1140cb8e0 2023-06-28T20:08:07.200 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.195+0000 7fd11c454700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1140cf260 con 0x7fd1140d10f0 2023-06-28T20:08:07.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.195+0000 7fd11c454700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1140cd1b0 con 0x7fd1140cedf0 2023-06-28T20:08:07.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.195+0000 7fd11a1f0700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd1140d10f0 0x7fd1140ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:07.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.195+0000 7fd11a1f0700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd1140d10f0 0x7fd1140ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54552/0 (socket says 172.21.15.43:54552) 2023-06-28T20:08:07.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.195+0000 7fd11a1f0700 1 -- 172.21.15.43:0/117165224 learned_addr learned my addr 172.21.15.43:0/117165224 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:07.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.196+0000 7fd11a1f0700 1 -- 172.21.15.43:0/117165224 >> v2:172.21.15.178:3300/0 conn(0x7fd1140cedf0 msgr2=0x7fd1140cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:07.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.196+0000 7fd1199ef700 1 --2- 172.21.15.43:0/117165224 >> v2:172.21.15.178:3300/0 conn(0x7fd1140cedf0 0x7fd1140cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:07.201 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.196+0000 7fd11a1f0700 1 --2- 172.21.15.43:0/117165224 >> v2:172.21.15.178:3300/0 conn(0x7fd1140cedf0 0x7fd1140cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.196+0000 7fd11a1f0700 1 -- 172.21.15.43:0/117165224 >> v1:172.21.15.178:6789/0 conn(0x7fd1140cb8e0 legacy=0x7fd1140cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:07.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.196+0000 7fd11a1f0700 1 -- 172.21.15.43:0/117165224 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1140cc110 con 0x7fd1140d10f0 2023-06-28T20:08:07.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.196+0000 7fd11a1f0700 1 --2- 172.21.15.43:0/117165224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1140d10f0 0x7fd1140ce8b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd11404e4a0 tx=0x7fd104009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67dbca937e34c29 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:07.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.196+0000 7fd1191ee700 1 -- 172.21.15.43:0/117165224 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd104005e30 con 0x7fd1140d10f0 2023-06-28T20:08:07.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.196+0000 7fd1191ee700 1 -- 172.21.15.43:0/117165224 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd104006030 con 0x7fd1140d10f0 2023-06-28T20:08:07.202 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.196+0000 7fd1191ee700 1 -- 172.21.15.43:0/117165224 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1040063e0 con 0x7fd1140d10f0 2023-06-28T20:08:07.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.197+0000 7fd11c454700 1 -- 172.21.15.43:0/117165224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1140d10f0 msgr2=0x7fd1140ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:07.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.197+0000 7fd11c454700 1 --2- 172.21.15.43:0/117165224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1140d10f0 0x7fd1140ce8b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd11404e4a0 tx=0x7fd104009bd0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.197+0000 7fd11c454700 1 -- 172.21.15.43:0/117165224 shutdown_connections 2023-06-28T20:08:07.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.197+0000 7fd11c454700 1 --2- 172.21.15.43:0/117165224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1140d10f0 0x7fd1140ce8b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.197+0000 7fd11c454700 1 --2- 172.21.15.43:0/117165224 >> v2:172.21.15.178:3300/0 conn(0x7fd1140cedf0 0x7fd1140cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.197+0000 7fd11c454700 1 -- 172.21.15.43:0/117165224 >> 172.21.15.43:0/117165224 conn(0x7fd1140664b0 msgr2=0x7fd11409a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:07.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.197+0000 7fd11c454700 1 -- 172.21.15.43:0/117165224 shutdown_connections 2023-06-28T20:08:07.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.197+0000 7fd11c454700 1 -- 172.21.15.43:0/117165224 wait complete. 2023-06-28T20:08:07.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.198+0000 7fd11c454700 1 Processor -- start 2023-06-28T20:08:07.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.198+0000 7fd11c454700 1 -- start start 2023-06-28T20:08:07.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.199+0000 7fd11c454700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1140cb8e0 0x7fd1141f3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:07.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.199+0000 7fd11c454700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1140cedf0 0x7fd1140aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:07.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.199+0000 7fd11c454700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1140d10f0 0x7fd114146be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:07.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.199+0000 7fd11c454700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1141f3f50 con 0x7fd1140cb8e0 2023-06-28T20:08:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.199+0000 7fd11c454700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1140b5840 con 0x7fd1140d10f0 2023-06-28T20:08:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.199+0000 7fd11c454700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1140b5b20 con 0x7fd1140cedf0 2023-06-28T20:08:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.199+0000 7fd11a1f0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1140cb8e0 0x7fd1141f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.199+0000 7fd11a1f0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1140cb8e0 0x7fd1141f3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54558/0 (socket says 172.21.15.43:54558) 2023-06-28T20:08:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.199+0000 7fd11a1f0700 1 -- 172.21.15.43:0/2164027977 learned_addr learned my addr 172.21.15.43:0/2164027977 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.199+0000 7fd11a1f0700 1 -- 172.21.15.43:0/2164027977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1140cedf0 msgr2=0x7fd1140aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.199+0000 7fd1199ef700 1 --2- 172.21.15.43:0/2164027977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1140cedf0 0x7fd1140aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:07.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.199+0000 7fd11a9f1700 1 --2- 172.21.15.43:0/2164027977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1140d10f0 0x7fd114146be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:07.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.199+0000 7fd11a1f0700 1 --2- 172.21.15.43:0/2164027977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1140cedf0 0x7fd1140aebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.199+0000 7fd11a1f0700 1 -- 172.21.15.43:0/2164027977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1140d10f0 msgr2=0x7fd114146be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:07.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.199+0000 7fd11a1f0700 1 --2- 172.21.15.43:0/2164027977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1140d10f0 0x7fd114146be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.199+0000 7fd11a1f0700 1 -- 172.21.15.43:0/2164027977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd104009500 con 0x7fd1140cb8e0 2023-06-28T20:08:07.208 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.199+0000 7fd11a9f1700 1 --2- 172.21.15.43:0/2164027977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1140d10f0 0x7fd114146be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:07.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.199+0000 7fd11a1f0700 1 --2- 172.21.15.43:0/2164027977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1140cb8e0 0x7fd1141f3a10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd104006000 tx=0x7fd104004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:07.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.199+0000 7fd10affd700 1 -- 172.21.15.43:0/2164027977 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd104004250 con 0x7fd1140cb8e0 2023-06-28T20:08:07.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.200+0000 7fd10affd700 1 -- 172.21.15.43:0/2164027977 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1040043b0 con 0x7fd1140cb8e0 2023-06-28T20:08:07.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.200+0000 7fd11c454700 1 -- 172.21.15.43:0/2164027977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1140b5e00 con 0x7fd1140cb8e0 2023-06-28T20:08:07.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.200+0000 7fd10affd700 1 -- 172.21.15.43:0/2164027977 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1040036a0 con 0x7fd1140cb8e0 2023-06-28T20:08:07.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.200+0000 7fd11c454700 1 -- 172.21.15.43:0/2164027977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1140b63c0 con 0x7fd1140cb8e0 2023-06-28T20:08:07.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.200+0000 7fd11c454700 1 -- 172.21.15.43:0/2164027977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1140728b0 con 0x7fd1140cb8e0 2023-06-28T20:08:07.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.201+0000 7fd10affd700 1 -- 172.21.15.43:0/2164027977 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd104022070 con 0x7fd1140cb8e0 2023-06-28T20:08:07.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.201+0000 7fd10affd700 1 --2- 172.21.15.43:0/2164027977 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd0f806ace0 0x7fd0f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:07.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.202+0000 7fd10affd700 1 -- 172.21.15.43:0/2164027977 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd10408b0e0 con 0x7fd1140cb8e0 2023-06-28T20:08:07.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.202+0000 7fd1199ef700 1 --2- 172.21.15.43:0/2164027977 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd0f806ace0 0x7fd0f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:07.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.202+0000 7fd1199ef700 1 --2- 172.21.15.43:0/2164027977 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd0f806ace0 0x7fd0f806d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd1140ce560 tx=0x7fd110008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:07.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.206+0000 7fd10affd700 1 -- 172.21.15.43:0/2164027977 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd104056180 con 0x7fd1140cb8e0 2023-06-28T20:08:07.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.449+0000 7fd11c454700 1 -- 172.21.15.43:0/2164027977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1140ccc80 con 0x7fd1140cb8e0 2023-06-28T20:08:07.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.450+0000 7fd10affd700 1 -- 172.21.15.43:0/2164027977 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4665 (secure 0 0 0) 0x7fd1040597a0 con 0x7fd1140cb8e0 2023-06-28T20:08:07.454 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:07.454 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4033115951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4033115951},{"type":"v1","addr":"172.21.15.178:6835","nonce":4033115951}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/65569869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":65569869},{"type":"v1","addr":"172.21.15.43:6837","nonce":65569869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/1016213057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016213057},{"type":"v1","addr":"172.21.15.178:6834","nonce":1016213057}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-28T20:07:20.788417+0000","modified":"2023-06-28T20:07:24.388714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/3514544484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3514544484},{"type":"v1","addr":"172.21.15.43:6835","nonce":3514544484}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-28T20:08:07.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.452+0000 7fd11c454700 1 -- 172.21.15.43:0/2164027977 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd0f806ace0 msgr2=0x7fd0f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:07.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.452+0000 7fd11c454700 1 --2- 172.21.15.43:0/2164027977 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd0f806ace0 0x7fd0f806d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd1140ce560 tx=0x7fd110008040 comp rx=0 tx=0).stop 2023-06-28T20:08:07.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.452+0000 7fd11c454700 1 -- 172.21.15.43:0/2164027977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1140cb8e0 msgr2=0x7fd1141f3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:07.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.452+0000 7fd11c454700 1 --2- 172.21.15.43:0/2164027977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1140cb8e0 0x7fd1141f3a10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd104006000 tx=0x7fd104004750 comp rx=0 tx=0).stop 2023-06-28T20:08:07.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.452+0000 7fd11c454700 1 -- 172.21.15.43:0/2164027977 shutdown_connections 2023-06-28T20:08:07.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.452+0000 7fd11c454700 1 --2- 172.21.15.43:0/2164027977 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1140d10f0 0x7fd114146be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.452+0000 7fd11c454700 1 --2- 172.21.15.43:0/2164027977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1140cb8e0 0x7fd1141f3a10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.452+0000 7fd11c454700 1 --2- 172.21.15.43:0/2164027977 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd0f806ace0 0x7fd0f806d190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.452+0000 7fd11c454700 1 --2- 172.21.15.43:0/2164027977 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1140cedf0 0x7fd1140aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.452+0000 7fd11c454700 1 -- 172.21.15.43:0/2164027977 >> 172.21.15.43:0/2164027977 conn(0x7fd1140664b0 msgr2=0x7fd11406e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:07.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.452+0000 7fd11c454700 1 -- 172.21.15.43:0/2164027977 shutdown_connections 2023-06-28T20:08:07.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.452+0000 7fd11c454700 1 -- 172.21.15.43:0/2164027977 wait complete. 2023-06-28T20:08:07.458 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 7 2023-06-28T20:08:07.469 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-28T20:08:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.590+0000 7f634924e700 1 Processor -- start 2023-06-28T20:08:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.590+0000 7f634924e700 1 -- start start 2023-06-28T20:08:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.591+0000 7f634924e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f634415d9f0 0x7f634406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.591+0000 7f634924e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6344070cb0 0x7f6344069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.591+0000 7f634924e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f634406a430 0x7f634415df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.591+0000 7f634924e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f634406a8d0 con 0x7f6344070cb0 2023-06-28T20:08:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.591+0000 7f634924e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f634415ddc0 con 0x7f634406a430 2023-06-28T20:08:07.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.591+0000 7f634924e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f634406f3e0 con 0x7f634415d9f0 2023-06-28T20:08:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.591+0000 7f634259c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6344070cb0 0x7f6344069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.591+0000 7f634259c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6344070cb0 0x7f6344069ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54566/0 (socket says 172.21.15.43:54566) 2023-06-28T20:08:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.591+0000 7f634259c700 1 -- 172.21.15.43:0/3048885232 learned_addr learned my addr 172.21.15.43:0/3048885232 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.592+0000 7f634259c700 1 -- 172.21.15.43:0/3048885232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f634415d9f0 msgr2=0x7f634406d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.592+0000 7f634359e700 1 --2- 172.21.15.43:0/3048885232 >> v2:172.21.15.178:3300/0 conn(0x7f634406a430 0x7f634415df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.592+0000 7f6342d9d700 1 --2- 172.21.15.43:0/3048885232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f634415d9f0 0x7f634406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:07.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.592+0000 7f634259c700 1 --2- 172.21.15.43:0/3048885232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f634415d9f0 0x7f634406d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.592+0000 7f634259c700 1 -- 172.21.15.43:0/3048885232 >> v2:172.21.15.178:3300/0 conn(0x7f634406a430 msgr2=0x7f634415df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.592+0000 7f634259c700 1 --2- 172.21.15.43:0/3048885232 >> v2:172.21.15.178:3300/0 conn(0x7f634406a430 0x7f634415df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.592+0000 7f634259c700 1 -- 172.21.15.43:0/3048885232 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6344071170 con 0x7f6344070cb0 2023-06-28T20:08:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.592+0000 7f6342d9d700 1 --2- 172.21.15.43:0/3048885232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f634415d9f0 0x7f634406d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.592+0000 7f634359e700 1 --2- 172.21.15.43:0/3048885232 >> v2:172.21.15.178:3300/0 conn(0x7f634406a430 0x7f634415df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.592+0000 7f634259c700 1 --2- 172.21.15.43:0/3048885232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6344070cb0 0x7f6344069ef0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6344071c00 tx=0x7f6338009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5d92cbcb2e4479 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.592+0000 7f6341d9b700 1 -- 172.21.15.43:0/3048885232 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633800b860 con 0x7f6344070cb0 2023-06-28T20:08:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.592+0000 7f6341d9b700 1 -- 172.21.15.43:0/3048885232 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633800b9c0 con 0x7f6344070cb0 2023-06-28T20:08:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.592+0000 7f6341d9b700 1 -- 172.21.15.43:0/3048885232 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633800bd50 con 0x7f6344070cb0 2023-06-28T20:08:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.592+0000 7f634924e700 1 -- 172.21.15.43:0/3048885232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6344070cb0 msgr2=0x7f6344069ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.592+0000 7f634924e700 1 --2- 172.21.15.43:0/3048885232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6344070cb0 0x7f6344069ef0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6344071c00 tx=0x7f6338009c00 comp rx=0 tx=0).stop 2023-06-28T20:08:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.592+0000 7f634924e700 1 -- 172.21.15.43:0/3048885232 shutdown_connections 2023-06-28T20:08:07.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.592+0000 7f634924e700 1 --2- 172.21.15.43:0/3048885232 >> v2:172.21.15.178:3300/0 conn(0x7f634406a430 0x7f634415df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.592+0000 7f634924e700 1 --2- 172.21.15.43:0/3048885232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f634415d9f0 0x7f634406d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.592+0000 7f634924e700 1 --2- 172.21.15.43:0/3048885232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6344070cb0 0x7f6344069ef0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.592+0000 7f634924e700 1 -- 172.21.15.43:0/3048885232 >> 172.21.15.43:0/3048885232 conn(0x7f63440b1720 msgr2=0x7f63440b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:07.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.593+0000 7f634924e700 1 -- 172.21.15.43:0/3048885232 shutdown_connections 2023-06-28T20:08:07.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.593+0000 7f634924e700 1 -- 172.21.15.43:0/3048885232 wait complete. 2023-06-28T20:08:07.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.594+0000 7f634924e700 1 Processor -- start 2023-06-28T20:08:07.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.594+0000 7f634924e700 1 -- start start 2023-06-28T20:08:07.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.595+0000 7f634924e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f634406a430 0x7f63440c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:07.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.595+0000 7f634924e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6344070cb0 0x7f63441596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:07.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.595+0000 7f634924e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634415d9f0 0x7f6344155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:07.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.595+0000 7f634924e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6344159e90 con 0x7f634415d9f0 2023-06-28T20:08:07.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.595+0000 7f634924e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6344159bf0 con 0x7f6344070cb0 2023-06-28T20:08:07.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.595+0000 7f634924e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63440c7d80 con 0x7f634406a430 2023-06-28T20:08:07.604 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.595+0000 7f634359e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634415d9f0 0x7f6344155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:07.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.595+0000 7f634359e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634415d9f0 0x7f6344155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54574/0 (socket says 172.21.15.43:54574) 2023-06-28T20:08:07.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.595+0000 7f634359e700 1 -- 172.21.15.43:0/1132597271 learned_addr learned my addr 172.21.15.43:0/1132597271 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:07.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.595+0000 7f634259c700 1 --2- 172.21.15.43:0/1132597271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6344070cb0 0x7f63441596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:07.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.595+0000 7f6342d9d700 1 --2- 172.21.15.43:0/1132597271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f634406a430 0x7f63440c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:07.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.595+0000 7f634359e700 1 -- 172.21.15.43:0/1132597271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f634406a430 msgr2=0x7f63440c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:07.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.595+0000 7f634359e700 1 --2- 172.21.15.43:0/1132597271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f634406a430 0x7f63440c99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.605 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.595+0000 7f634359e700 1 -- 172.21.15.43:0/1132597271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6344070cb0 msgr2=0x7f63441596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:07.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.595+0000 7f634359e700 1 --2- 172.21.15.43:0/1132597271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6344070cb0 0x7f63441596b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.595+0000 7f634359e700 1 -- 172.21.15.43:0/1132597271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6338009550 con 0x7f634415d9f0 2023-06-28T20:08:07.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.595+0000 7f634359e700 1 --2- 172.21.15.43:0/1132597271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634415d9f0 0x7f6344155a70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f633400c7f0 tx=0x7f633400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:07.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.596+0000 7f632b7fe700 1 -- 172.21.15.43:0/1132597271 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6334011800 con 0x7f634415d9f0 2023-06-28T20:08:07.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.596+0000 7f632b7fe700 1 -- 172.21.15.43:0/1132597271 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6334011980 con 0x7f634415d9f0 2023-06-28T20:08:07.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.596+0000 7f634924e700 1 -- 172.21.15.43:0/1132597271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63440c8060 con 0x7f634415d9f0 2023-06-28T20:08:07.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.596+0000 7f632b7fe700 1 -- 172.21.15.43:0/1132597271 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6334011d30 con 0x7f634415d9f0 2023-06-28T20:08:07.606 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.596+0000 7f634924e700 1 -- 172.21.15.43:0/1132597271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63440c8620 con 0x7f634415d9f0 2023-06-28T20:08:07.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.597+0000 7f634924e700 1 -- 172.21.15.43:0/1132597271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63440c8810 con 0x7f634415d9f0 2023-06-28T20:08:07.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.597+0000 7f632b7fe700 1 -- 172.21.15.43:0/1132597271 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6334022350 con 0x7f634415d9f0 2023-06-28T20:08:07.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.597+0000 7f632b7fe700 1 --2- 172.21.15.43:0/1132597271 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f633006ace0 0x7f633006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.598+0000 7f632b7fe700 1 -- 172.21.15.43:0/1132597271 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6334089460 con 0x7f634415d9f0 2023-06-28T20:08:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.598+0000 7f6342d9d700 1 --2- 172.21.15.43:0/1132597271 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f633006ace0 0x7f633006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.598+0000 7f6342d9d700 1 --2- 172.21.15.43:0/1132597271 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f633006ace0 0x7f633006d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6344158410 tx=0x7f633c008450 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:07.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.602+0000 7f632b7fe700 1 -- 172.21.15.43:0/1132597271 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6334057fb0 con 0x7f634415d9f0 2023-06-28T20:08:07.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.847+0000 7f634924e700 1 -- 172.21.15.43:0/1132597271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63440728b0 con 0x7f634415d9f0 2023-06-28T20:08:07.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.848+0000 7f632b7fe700 1 -- 172.21.15.43:0/1132597271 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4665 (secure 0 0 0) 0x7f6334057b40 con 0x7f634415d9f0 2023-06-28T20:08:07.852 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:07.852 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/4033115951","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":4033115951},{"type":"v1","addr":"172.21.15.178:6835","nonce":4033115951}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/65569869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6836","nonce":65569869},{"type":"v1","addr":"172.21.15.43:6837","nonce":65569869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/1016213057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":1016213057},{"type":"v1","addr":"172.21.15.178:6834","nonce":1016213057}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-28T20:07:20.788417+0000","modified":"2023-06-28T20:07:24.388714+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/3514544484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":3514544484},{"type":"v1","addr":"172.21.15.43:6835","nonce":3514544484}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-28T20:08:07.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.850+0000 7f634924e700 1 -- 172.21.15.43:0/1132597271 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f633006ace0 msgr2=0x7f633006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:07.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.850+0000 7f634924e700 1 --2- 172.21.15.43:0/1132597271 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f633006ace0 0x7f633006d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6344158410 tx=0x7f633c008450 comp rx=0 tx=0).stop 2023-06-28T20:08:07.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.850+0000 7f634924e700 1 -- 172.21.15.43:0/1132597271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634415d9f0 msgr2=0x7f6344155a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:07.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.850+0000 7f634924e700 1 --2- 172.21.15.43:0/1132597271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634415d9f0 0x7f6344155a70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f633400c7f0 tx=0x7f633400cbb0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.850+0000 7f634924e700 1 -- 172.21.15.43:0/1132597271 shutdown_connections 2023-06-28T20:08:07.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.850+0000 7f634924e700 1 --2- 172.21.15.43:0/1132597271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f634415d9f0 0x7f6344155a70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.850+0000 7f634924e700 1 --2- 172.21.15.43:0/1132597271 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f633006ace0 0x7f633006d190 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.850+0000 7f634924e700 1 --2- 172.21.15.43:0/1132597271 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f634406a430 0x7f63440c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.850+0000 7f634924e700 1 --2- 172.21.15.43:0/1132597271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6344070cb0 0x7f63441596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.850+0000 7f634924e700 1 -- 172.21.15.43:0/1132597271 >> 172.21.15.43:0/1132597271 conn(0x7f63440b1720 msgr2=0x7f63440b0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:07.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.850+0000 7f634924e700 1 -- 172.21.15.43:0/1132597271 shutdown_connections 2023-06-28T20:08:07.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.850+0000 7f634924e700 1 -- 172.21.15.43:0/1132597271 wait complete. 2023-06-28T20:08:07.856 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 7 2023-06-28T20:08:07.867 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:08:07.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.986+0000 7fc44078d700 1 Processor -- start 2023-06-28T20:08:07.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.986+0000 7fc44078d700 1 -- start start 2023-06-28T20:08:07.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.987+0000 7fc44078d700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc4380d10f0 0x7fc4380ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:07.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.987+0000 7fc44078d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4380cbab0 0x7fc4380b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:07.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.987+0000 7fc44078d700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc43815ab50 con 0x7fc4380cedf0 2023-06-28T20:08:07.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.987+0000 7fc44078d700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4380cbf30 con 0x7fc4380d10f0 2023-06-28T20:08:07.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.987+0000 7fc44078d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4380cf260 con 0x7fc4380cbab0 2023-06-28T20:08:07.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.988+0000 7fc43e529700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fc4380d10f0 0x7fc4380ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:07.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.988+0000 7fc43ed2a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4380cbab0 0x7fc4380b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:07.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.988+0000 7fc43dd28700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7fc4380cedf0 0x7fc4380cb3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:42890/0 (socket says 172.21.15.43:42890) 2023-06-28T20:08:07.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.988+0000 7fc43dd28700 1 -- 172.21.15.43:0/90645275 learned_addr learned my addr 172.21.15.43:0/90645275 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:07.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.988+0000 7fc43d527700 1 -- 172.21.15.43:0/90645275 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2206145931 0 0) 0x7fc43815ab50 con 0x7fc4380cedf0 2023-06-28T20:08:07.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.988+0000 7fc43d527700 1 -- 172.21.15.43:0/90645275 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4240035b0 con 0x7fc4380cedf0 2023-06-28T20:08:07.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.988+0000 7fc43ed2a700 1 -- 172.21.15.43:0/90645275 >> v2:172.21.15.178:3300/0 conn(0x7fc4380d10f0 msgr2=0x7fc4380ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:07.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.988+0000 7fc43ed2a700 1 --2- 172.21.15.43:0/90645275 >> v2:172.21.15.178:3300/0 conn(0x7fc4380d10f0 0x7fc4380ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.988+0000 7fc43ed2a700 1 -- 172.21.15.43:0/90645275 >> v1:172.21.15.178:6789/0 conn(0x7fc4380cedf0 legacy=0x7fc4380cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:07.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.988+0000 7fc43ed2a700 1 -- 172.21.15.43:0/90645275 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4380cd1b0 con 0x7fc4380cbab0 2023-06-28T20:08:07.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.988+0000 7fc43e529700 1 --2- 172.21.15.43:0/90645275 >> v2:172.21.15.178:3300/0 conn(0x7fc4380d10f0 0x7fc4380ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:08:07.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.988+0000 7fc43ed2a700 1 --2- 172.21.15.43:0/90645275 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4380cbab0 0x7fc4380b0600 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc430009d90 tx=0x7fc43000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22c26dba8a5ace5d server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:07.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.989+0000 7fc43d527700 1 -- 172.21.15.43:0/90645275 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43000e070 con 0x7fc4380cbab0 2023-06-28T20:08:07.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.989+0000 7fc43d527700 1 -- 172.21.15.43:0/90645275 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43000bb20 con 0x7fc4380cbab0 2023-06-28T20:08:07.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.989+0000 7fc43d527700 1 -- 172.21.15.43:0/90645275 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc430012430 con 0x7fc4380cbab0 2023-06-28T20:08:07.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.989+0000 7fc44078d700 1 -- 172.21.15.43:0/90645275 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4380cbab0 msgr2=0x7fc4380b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:07.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.989+0000 7fc44078d700 1 --2- 172.21.15.43:0/90645275 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4380cbab0 0x7fc4380b0600 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc430009d90 tx=0x7fc43000b330 comp rx=0 tx=0).stop 2023-06-28T20:08:07.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.989+0000 7fc44078d700 1 -- 172.21.15.43:0/90645275 shutdown_connections 2023-06-28T20:08:07.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.989+0000 7fc44078d700 1 --2- 172.21.15.43:0/90645275 >> v2:172.21.15.178:3300/0 conn(0x7fc4380d10f0 0x7fc4380ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.989+0000 7fc44078d700 1 --2- 172.21.15.43:0/90645275 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4380cbab0 0x7fc4380b0600 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:07.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.989+0000 7fc44078d700 1 -- 172.21.15.43:0/90645275 >> 172.21.15.43:0/90645275 conn(0x7fc4380664b0 msgr2=0x7fc43809a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:07.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.989+0000 7fc44078d700 1 -- 172.21.15.43:0/90645275 shutdown_connections 2023-06-28T20:08:07.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.989+0000 7fc44078d700 1 -- 172.21.15.43:0/90645275 wait complete. 2023-06-28T20:08:08.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.990+0000 7fc44078d700 1 Processor -- start 2023-06-28T20:08:08.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.990+0000 7fc44078d700 1 -- start start 2023-06-28T20:08:08.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.991+0000 7fc44078d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4380cbab0 0x7fc4381ef4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:08.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.991+0000 7fc44078d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4380cedf0 0x7fc438158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:08.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.991+0000 7fc44078d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4380d10f0 0x7fc438159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:08.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.991+0000 7fc44078d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4380c92f0 con 0x7fc4380cbab0 2023-06-28T20:08:08.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.991+0000 7fc44078d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4380c8d30 con 0x7fc4380cedf0 2023-06-28T20:08:08.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.991+0000 7fc44078d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4380c9010 con 0x7fc4380d10f0 2023-06-28T20:08:08.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.991+0000 7fc43e529700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4380cbab0 0x7fc4381ef4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:08.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.991+0000 7fc43e529700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4380cbab0 0x7fc4381ef4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54586/0 (socket says 172.21.15.43:54586) 2023-06-28T20:08:08.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.991+0000 7fc43e529700 1 -- 172.21.15.43:0/953065656 learned_addr learned my addr 172.21.15.43:0/953065656 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:08.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.991+0000 7fc43dd28700 1 --2- 172.21.15.43:0/953065656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4380cedf0 0x7fc438158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:08.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.991+0000 7fc43ed2a700 1 --2- 172.21.15.43:0/953065656 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4380d10f0 0x7fc438159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:08.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.992+0000 7fc43e529700 1 -- 172.21.15.43:0/953065656 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4380d10f0 msgr2=0x7fc438159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:08.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.992+0000 7fc43e529700 1 --2- 172.21.15.43:0/953065656 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4380d10f0 0x7fc438159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:08.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.992+0000 7fc43e529700 1 -- 172.21.15.43:0/953065656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4380cedf0 msgr2=0x7fc438158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:08.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.992+0000 7fc43e529700 1 --2- 172.21.15.43:0/953065656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4380cedf0 0x7fc438158bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:08.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.992+0000 7fc43e529700 1 -- 172.21.15.43:0/953065656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc434009430 con 0x7fc4380cbab0 2023-06-28T20:08:08.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.992+0000 7fc43dd28700 1 --2- 172.21.15.43:0/953065656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4380cedf0 0x7fc438158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:08.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.992+0000 7fc43e529700 1 --2- 172.21.15.43:0/953065656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4380cbab0 0x7fc4381ef4f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc43400ebe0 tx=0x7fc434009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:08.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.992+0000 7fc42affd700 1 -- 172.21.15.43:0/953065656 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434005a20 con 0x7fc4380cbab0 2023-06-28T20:08:08.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.992+0000 7fc42affd700 1 -- 172.21.15.43:0/953065656 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc434005b80 con 0x7fc4380cbab0 2023-06-28T20:08:08.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.992+0000 7fc44078d700 1 -- 172.21.15.43:0/953065656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc430009a70 con 0x7fc4380cbab0 2023-06-28T20:08:08.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.992+0000 7fc42affd700 1 -- 172.21.15.43:0/953065656 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43400c8b0 con 0x7fc4380cbab0 2023-06-28T20:08:08.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.992+0000 7fc44078d700 1 -- 172.21.15.43:0/953065656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4381451c0 con 0x7fc4380cbab0 2023-06-28T20:08:08.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.993+0000 7fc44078d700 1 -- 172.21.15.43:0/953065656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc438144d40 con 0x7fc4380cbab0 2023-06-28T20:08:08.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.993+0000 7fc42affd700 1 -- 172.21.15.43:0/953065656 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc43400ca10 con 0x7fc4380cbab0 2023-06-28T20:08:08.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.994+0000 7fc42affd700 1 --2- 172.21.15.43:0/953065656 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc41c06ace0 0x7fc41c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:08.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.994+0000 7fc42affd700 1 -- 172.21.15.43:0/953065656 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc434089750 con 0x7fc4380cbab0 2023-06-28T20:08:08.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.994+0000 7fc43dd28700 1 --2- 172.21.15.43:0/953065656 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc41c06ace0 0x7fc41c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:08.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.994+0000 7fc43dd28700 1 --2- 172.21.15.43:0/953065656 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc41c06ace0 0x7fc41c06d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc43806edb0 tx=0x7fc42c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:08.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:07.998+0000 7fc42affd700 1 -- 172.21.15.43:0/953065656 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc438144d40 con 0x7fc4380cbab0 2023-06-28T20:08:08.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.207+0000 7fc44078d700 1 -- 172.21.15.43:0/953065656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc438144d40 con 0x7fc4380cbab0 2023-06-28T20:08:08.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.208+0000 7fc42affd700 1 -- 172.21.15.43:0/953065656 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+18317 (secure 0 0 0) 0x7fc438144d40 con 0x7fc4380cbab0 2023-06-28T20:08:08.213 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:08.213 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":22,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:08:08.099834+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:07:18.624759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-28T20:07:19.760017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.37","mappings":[{"from":3,"to":2}]},{"pgid":"4.3e","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:08:08.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.210+0000 7fc44078d700 1 -- 172.21.15.43:0/953065656 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc41c06ace0 msgr2=0x7fc41c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:08.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.210+0000 7fc44078d700 1 --2- 172.21.15.43:0/953065656 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc41c06ace0 0x7fc41c06d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc43806edb0 tx=0x7fc42c009040 comp rx=0 tx=0).stop 2023-06-28T20:08:08.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.210+0000 7fc44078d700 1 -- 172.21.15.43:0/953065656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4380cbab0 msgr2=0x7fc4381ef4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:08.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.210+0000 7fc44078d700 1 --2- 172.21.15.43:0/953065656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4380cbab0 0x7fc4381ef4f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc43400ebe0 tx=0x7fc434009e00 comp rx=0 tx=0).stop 2023-06-28T20:08:08.215 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.210+0000 7fc44078d700 1 -- 172.21.15.43:0/953065656 shutdown_connections 2023-06-28T20:08:08.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.210+0000 7fc44078d700 1 --2- 172.21.15.43:0/953065656 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fc41c06ace0 0x7fc41c06d190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:08.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.210+0000 7fc44078d700 1 --2- 172.21.15.43:0/953065656 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fc4380d10f0 0x7fc438159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:08.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.210+0000 7fc44078d700 1 --2- 172.21.15.43:0/953065656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4380cbab0 0x7fc4381ef4f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:08.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.210+0000 7fc44078d700 1 --2- 172.21.15.43:0/953065656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc4380cedf0 0x7fc438158bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:08.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.210+0000 7fc44078d700 1 -- 172.21.15.43:0/953065656 >> 172.21.15.43:0/953065656 conn(0x7fc4380664b0 msgr2=0x7fc43809a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:08.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.210+0000 7fc44078d700 1 -- 172.21.15.43:0/953065656 shutdown_connections 2023-06-28T20:08:08.217 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.210+0000 7fc44078d700 1 -- 172.21.15.43:0/953065656 wait complete. 2023-06-28T20:08:08.227 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-06-28T20:08:08.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.348+0000 7f9ee820c700 1 Processor -- start 2023-06-28T20:08:08.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.349+0000 7f9ee820c700 1 -- start start 2023-06-28T20:08:08.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.349+0000 7f9ee820c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9ee0157ec0 0x7f9ee00b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:08.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.349+0000 7f9ee820c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ee00b1dc0 0x7f9ee0070900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:08.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.349+0000 7f9ee820c700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9ee0070e40 0x7f9ee0069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:08.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.349+0000 7f9ee820c700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee00b2260 con 0x7f9ee0157ec0 2023-06-28T20:08:08.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.349+0000 7f9ee820c700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee015a920 con 0x7f9ee0070e40 2023-06-28T20:08:08.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.349+0000 7f9ee820c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee0158290 con 0x7f9ee00b1dc0 2023-06-28T20:08:08.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.349+0000 7f9ee5fa8700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9ee0157ec0 0x7f9ee00b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:08.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.350+0000 7f9ee5fa8700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9ee0157ec0 0x7f9ee00b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54598/0 (socket says 172.21.15.43:54598) 2023-06-28T20:08:08.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.350+0000 7f9ee5fa8700 1 -- 172.21.15.43:0/480237546 learned_addr learned my addr 172.21.15.43:0/480237546 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:08.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.350+0000 7f9ee67a9700 1 --2- 172.21.15.43:0/480237546 >> v2:172.21.15.178:3300/0 conn(0x7f9ee0070e40 0x7f9ee0069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:08.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.350+0000 7f9ee57a7700 1 --2- 172.21.15.43:0/480237546 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ee00b1dc0 0x7f9ee0070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:08.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.350+0000 7f9ee5fa8700 1 -- 172.21.15.43:0/480237546 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ee00b1dc0 msgr2=0x7f9ee0070900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:08.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.350+0000 7f9ee5fa8700 1 --2- 172.21.15.43:0/480237546 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ee00b1dc0 0x7f9ee0070900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:08.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.350+0000 7f9ee5fa8700 1 -- 172.21.15.43:0/480237546 >> v2:172.21.15.178:3300/0 conn(0x7f9ee0070e40 msgr2=0x7f9ee0069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:08.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.350+0000 7f9ee5fa8700 1 --2- 172.21.15.43:0/480237546 >> v2:172.21.15.178:3300/0 conn(0x7f9ee0070e40 0x7f9ee0069ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:08.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.350+0000 7f9ee5fa8700 1 -- 172.21.15.43:0/480237546 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee015ab30 con 0x7f9ee0157ec0 2023-06-28T20:08:08.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.350+0000 7f9ee5fa8700 1 --2- 172.21.15.43:0/480237546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0157ec0 0x7f9ee00b1880 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9ee006bfc0 tx=0x7f9ed0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9face894a301e967 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:08.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.350+0000 7f9ee4fa6700 1 -- 172.21.15.43:0/480237546 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0011830 con 0x7f9ee0157ec0 2023-06-28T20:08:08.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.350+0000 7f9ee4fa6700 1 -- 172.21.15.43:0/480237546 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed0011990 con 0x7f9ee0157ec0 2023-06-28T20:08:08.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.350+0000 7f9ee4fa6700 1 -- 172.21.15.43:0/480237546 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0011d20 con 0x7f9ee0157ec0 2023-06-28T20:08:08.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.351+0000 7f9ee820c700 1 -- 172.21.15.43:0/480237546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0157ec0 msgr2=0x7f9ee00b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:08.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.351+0000 7f9ee820c700 1 --2- 172.21.15.43:0/480237546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0157ec0 0x7f9ee00b1880 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f9ee006bfc0 tx=0x7f9ed0009bd0 comp rx=0 tx=0).stop 2023-06-28T20:08:08.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.351+0000 7f9ee820c700 1 -- 172.21.15.43:0/480237546 shutdown_connections 2023-06-28T20:08:08.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.351+0000 7f9ee820c700 1 --2- 172.21.15.43:0/480237546 >> v2:172.21.15.178:3300/0 conn(0x7f9ee0070e40 0x7f9ee0069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:08.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.351+0000 7f9ee820c700 1 --2- 172.21.15.43:0/480237546 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0157ec0 0x7f9ee00b1880 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:08.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.351+0000 7f9ee820c700 1 --2- 172.21.15.43:0/480237546 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ee00b1dc0 0x7f9ee0070900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:08.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.351+0000 7f9ee820c700 1 -- 172.21.15.43:0/480237546 >> 172.21.15.43:0/480237546 conn(0x7f9ee006b000 msgr2=0x7f9ee006b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:08.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.351+0000 7f9ee820c700 1 -- 172.21.15.43:0/480237546 shutdown_connections 2023-06-28T20:08:08.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.351+0000 7f9ee820c700 1 -- 172.21.15.43:0/480237546 wait complete. 2023-06-28T20:08:08.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.352+0000 7f9ee820c700 1 Processor -- start 2023-06-28T20:08:08.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.353+0000 7f9ee820c700 1 -- start start 2023-06-28T20:08:08.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.353+0000 7f9ee820c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ee0070e40 0x7f9ee00c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:08.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.353+0000 7f9ee820c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ee00b1dc0 0x7f9ee0158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:08.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.353+0000 7f9ee820c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0157ec0 0x7f9ee00ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:08.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.353+0000 7f9ee820c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee00ad280 con 0x7f9ee0157ec0 2023-06-28T20:08:08.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.353+0000 7f9ee820c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee00ae0a0 con 0x7f9ee0070e40 2023-06-28T20:08:08.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.353+0000 7f9ee820c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee00ad490 con 0x7f9ee00b1dc0 2023-06-28T20:08:08.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.354+0000 7f9ee67a9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0157ec0 0x7f9ee00ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:08.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.354+0000 7f9ee67a9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0157ec0 0x7f9ee00ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54608/0 (socket says 172.21.15.43:54608) 2023-06-28T20:08:08.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.354+0000 7f9ee67a9700 1 -- 172.21.15.43:0/2403955196 learned_addr learned my addr 172.21.15.43:0/2403955196 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:08.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.354+0000 7f9ee5fa8700 1 --2- 172.21.15.43:0/2403955196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ee0070e40 0x7f9ee00c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:08.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.354+0000 7f9ee57a7700 1 --2- 172.21.15.43:0/2403955196 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ee00b1dc0 0x7f9ee0158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:08.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.354+0000 7f9ee67a9700 1 -- 172.21.15.43:0/2403955196 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ee00b1dc0 msgr2=0x7f9ee0158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:08.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.354+0000 7f9ee67a9700 1 --2- 172.21.15.43:0/2403955196 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ee00b1dc0 0x7f9ee0158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:08.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.354+0000 7f9ee67a9700 1 -- 172.21.15.43:0/2403955196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ee0070e40 msgr2=0x7f9ee00c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:08.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.354+0000 7f9ee67a9700 1 --2- 172.21.15.43:0/2403955196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ee0070e40 0x7f9ee00c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:08.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.354+0000 7f9ee67a9700 1 -- 172.21.15.43:0/2403955196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed0009550 con 0x7f9ee0157ec0 2023-06-28T20:08:08.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.354+0000 7f9ee67a9700 1 --2- 172.21.15.43:0/2403955196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0157ec0 0x7f9ee00ce060 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800eed0 tx=0x7f9ed800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:08.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.354+0000 7f9ed67fc700 1 -- 172.21.15.43:0/2403955196 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8010040 con 0x7f9ee0157ec0 2023-06-28T20:08:08.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.354+0000 7f9ed67fc700 1 -- 172.21.15.43:0/2403955196 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed800bc80 con 0x7f9ee0157ec0 2023-06-28T20:08:08.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.354+0000 7f9ed67fc700 1 -- 172.21.15.43:0/2403955196 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8009e40 con 0x7f9ee0157ec0 2023-06-28T20:08:08.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.354+0000 7f9ee820c700 1 -- 172.21.15.43:0/2403955196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee00ad770 con 0x7f9ee0157ec0 2023-06-28T20:08:08.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.354+0000 7f9ee820c700 1 -- 172.21.15.43:0/2403955196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee00add30 con 0x7f9ee0157ec0 2023-06-28T20:08:08.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.355+0000 7f9ee820c700 1 -- 172.21.15.43:0/2403955196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee00728b0 con 0x7f9ee0157ec0 2023-06-28T20:08:08.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.356+0000 7f9ed67fc700 1 -- 172.21.15.43:0/2403955196 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9ed801d400 con 0x7f9ee0157ec0 2023-06-28T20:08:08.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.357+0000 7f9ed67fc700 1 --2- 172.21.15.43:0/2403955196 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9ec406ad10 0x7f9ec406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:08.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.357+0000 7f9ed67fc700 1 -- 172.21.15.43:0/2403955196 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9ed808a0a0 con 0x7f9ee0157ec0 2023-06-28T20:08:08.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.357+0000 7f9ee5fa8700 1 --2- 172.21.15.43:0/2403955196 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9ec406ad10 0x7f9ec406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:08.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.357+0000 7f9ee5fa8700 1 --2- 172.21.15.43:0/2403955196 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9ec406ad10 0x7f9ec406d1c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0014040 tx=0x7f9ed0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:08.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.360+0000 7f9ed67fc700 1 -- 172.21.15.43:0/2403955196 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9ed80551f0 con 0x7f9ee0157ec0 2023-06-28T20:08:08.612 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.608+0000 7f9ee820c700 1 -- 172.21.15.43:0/2403955196 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9ee00ae1e0 con 0x7f9ee0157ec0 2023-06-28T20:08:08.613 INFO:tasks.ceph.mon.a.smithi043.stderr:2023-06-28T20:08:08.609+0000 7fdb92760700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-06-28T20:08:08.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.613+0000 7f9ed67fc700 1 -- 172.21.15.43:0/2403955196 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9ed8058810 con 0x7f9ee0157ec0 2023-06-28T20:08:08.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.616+0000 7f9ee820c700 1 -- 172.21.15.43:0/2403955196 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9ec406ad10 msgr2=0x7f9ec406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:08.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.616+0000 7f9ee820c700 1 --2- 172.21.15.43:0/2403955196 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9ec406ad10 0x7f9ec406d1c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0014040 tx=0x7f9ed0008030 comp rx=0 tx=0).stop 2023-06-28T20:08:08.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.616+0000 7f9ee820c700 1 -- 172.21.15.43:0/2403955196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0157ec0 msgr2=0x7f9ee00ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:08.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.616+0000 7f9ee820c700 1 --2- 172.21.15.43:0/2403955196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0157ec0 0x7f9ee00ce060 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800eed0 tx=0x7f9ed800b390 comp rx=0 tx=0).stop 2023-06-28T20:08:08.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.619+0000 7f9ee820c700 1 -- 172.21.15.43:0/2403955196 shutdown_connections 2023-06-28T20:08:08.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.619+0000 7f9ee820c700 1 --2- 172.21.15.43:0/2403955196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0157ec0 0x7f9ee00ce060 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:08.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.619+0000 7f9ee820c700 1 --2- 172.21.15.43:0/2403955196 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ee0070e40 0x7f9ee00c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:08.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.619+0000 7f9ee820c700 1 --2- 172.21.15.43:0/2403955196 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9ec406ad10 0x7f9ec406d1c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:08.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.619+0000 7f9ee820c700 1 --2- 172.21.15.43:0/2403955196 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9ee00b1dc0 0x7f9ee0158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:08.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.619+0000 7f9ee820c700 1 -- 172.21.15.43:0/2403955196 >> 172.21.15.43:0/2403955196 conn(0x7f9ee006b000 msgr2=0x7f9ee00ca4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:08.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.619+0000 7f9ee820c700 1 -- 172.21.15.43:0/2403955196 shutdown_connections 2023-06-28T20:08:08.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.619+0000 7f9ee820c700 1 -- 172.21.15.43:0/2403955196 wait complete. 2023-06-28T20:08:08.625 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-06-28T20:08:08.636 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-06-28T20:08:08.660 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2023-06-28T20:08:08.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.762+0000 7f280f275700 1 Processor -- start 2023-06-28T20:08:08.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.763+0000 7f280f275700 1 -- start start 2023-06-28T20:08:08.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.763+0000 7f280f275700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f28080ceee0 0x7f280806d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:08.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.763+0000 7f280f275700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f28080afdf0 0x7f280806d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:08.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.763+0000 7f280f275700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28080ae950 con 0x7f280815a5f0 2023-06-28T20:08:08.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.763+0000 7f280f275700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f280815a9c0 con 0x7f28080afdf0 2023-06-28T20:08:08.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.763+0000 7f280f275700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28080add40 con 0x7f28080ceee0 2023-06-28T20:08:08.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.764+0000 7f280d812700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f28080afdf0 0x7f280806d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:08.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.764+0000 7f280d812700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f28080afdf0 0x7f280806d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54628/0 (socket says 172.21.15.43:54628) 2023-06-28T20:08:08.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.764+0000 7f280d812700 1 -- 172.21.15.43:0/701876225 learned_addr learned my addr 172.21.15.43:0/701876225 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:08.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.764+0000 7f280c810700 1 --2- 172.21.15.43:0/701876225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f28080ceee0 0x7f280806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:08.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.764+0000 7f280d812700 1 -- 172.21.15.43:0/701876225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f28080ceee0 msgr2=0x7f280806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:08.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.764+0000 7f280d812700 1 --2- 172.21.15.43:0/701876225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f28080ceee0 0x7f280806d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:08.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.764+0000 7f280d812700 1 -- 172.21.15.43:0/701876225 >> v1:172.21.15.178:6789/0 conn(0x7f280815a5f0 legacy=0x7f28080ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:08.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.764+0000 7f280d812700 1 -- 172.21.15.43:0/701876225 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28080ae020 con 0x7f28080afdf0 2023-06-28T20:08:08.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.764+0000 7f280c810700 1 --2- 172.21.15.43:0/701876225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f28080ceee0 0x7f280806d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:08.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.764+0000 7f280d812700 1 --2- 172.21.15.43:0/701876225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28080afdf0 0x7f280806d850 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2800009e50 tx=0x7f280000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a08fa151e92adab server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:08.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.764+0000 7f27fffff700 1 -- 172.21.15.43:0/701876225 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280000e070 con 0x7f28080afdf0 2023-06-28T20:08:08.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.764+0000 7f27fffff700 1 -- 172.21.15.43:0/701876225 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2800003a20 con 0x7f28080afdf0 2023-06-28T20:08:08.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.764+0000 7f27fffff700 1 -- 172.21.15.43:0/701876225 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2800003dd0 con 0x7f28080afdf0 2023-06-28T20:08:08.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.765+0000 7f280f275700 1 -- 172.21.15.43:0/701876225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28080afdf0 msgr2=0x7f280806d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:08.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.765+0000 7f280f275700 1 --2- 172.21.15.43:0/701876225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28080afdf0 0x7f280806d850 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2800009e50 tx=0x7f280000b330 comp rx=0 tx=0).stop 2023-06-28T20:08:08.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.765+0000 7f280f275700 1 -- 172.21.15.43:0/701876225 shutdown_connections 2023-06-28T20:08:08.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.765+0000 7f280f275700 1 --2- 172.21.15.43:0/701876225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28080afdf0 0x7f280806d850 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:08.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.765+0000 7f280f275700 1 --2- 172.21.15.43:0/701876225 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f28080ceee0 0x7f280806d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:08.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.765+0000 7f280f275700 1 -- 172.21.15.43:0/701876225 >> 172.21.15.43:0/701876225 conn(0x7f28080664b0 msgr2=0x7f28080d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:08.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.765+0000 7f280f275700 1 -- 172.21.15.43:0/701876225 shutdown_connections 2023-06-28T20:08:08.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.765+0000 7f280f275700 1 -- 172.21.15.43:0/701876225 wait complete. 2023-06-28T20:08:08.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.766+0000 7f280f275700 1 Processor -- start 2023-06-28T20:08:08.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.766+0000 7f280f275700 1 -- start start 2023-06-28T20:08:08.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.767+0000 7f280f275700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28080afdf0 0x7f28080b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:08.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.767+0000 7f280f275700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28080ceee0 0x7f28080c47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:08.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.767+0000 7f280f275700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f280815a5f0 0x7f2808143800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:08.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.767+0000 7f280f275700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28080c5d30 con 0x7f28080ceee0 2023-06-28T20:08:08.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.767+0000 7f280f275700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280815cb20 con 0x7f28080afdf0 2023-06-28T20:08:08.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.767+0000 7f280f275700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280815ce00 con 0x7f280815a5f0 2023-06-28T20:08:08.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.767+0000 7f280c810700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28080ceee0 0x7f28080c47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:08.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.767+0000 7f280c810700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28080ceee0 0x7f28080c47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54632/0 (socket says 172.21.15.43:54632) 2023-06-28T20:08:08.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.767+0000 7f280c810700 1 -- 172.21.15.43:0/1285326320 learned_addr learned my addr 172.21.15.43:0/1285326320 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:08.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.767+0000 7f280d812700 1 --2- 172.21.15.43:0/1285326320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f280815a5f0 0x7f2808143800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:08.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.767+0000 7f280d011700 1 --2- 172.21.15.43:0/1285326320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28080afdf0 0x7f28080b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:08.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.767+0000 7f280c810700 1 -- 172.21.15.43:0/1285326320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f280815a5f0 msgr2=0x7f2808143800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:08.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.767+0000 7f280c810700 1 --2- 172.21.15.43:0/1285326320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f280815a5f0 0x7f2808143800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:08.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.767+0000 7f280c810700 1 -- 172.21.15.43:0/1285326320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28080afdf0 msgr2=0x7f28080b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:08.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.767+0000 7f280c810700 1 --2- 172.21.15.43:0/1285326320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28080afdf0 0x7f28080b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:08.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.767+0000 7f280c810700 1 -- 172.21.15.43:0/1285326320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2800009b30 con 0x7f28080ceee0 2023-06-28T20:08:08.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.767+0000 7f280c810700 1 --2- 172.21.15.43:0/1285326320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28080ceee0 0x7f28080c47a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f280400c7f0 tx=0x7f280400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:08.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.767+0000 7f27fdffb700 1 -- 172.21.15.43:0/1285326320 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804011800 con 0x7f28080ceee0 2023-06-28T20:08:08.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.768+0000 7f27fdffb700 1 -- 172.21.15.43:0/1285326320 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2804011960 con 0x7f28080ceee0 2023-06-28T20:08:08.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.768+0000 7f280f275700 1 -- 172.21.15.43:0/1285326320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f280815d0e0 con 0x7f28080ceee0 2023-06-28T20:08:08.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.768+0000 7f27fdffb700 1 -- 172.21.15.43:0/1285326320 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804011cf0 con 0x7f28080ceee0 2023-06-28T20:08:08.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.768+0000 7f280f275700 1 -- 172.21.15.43:0/1285326320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2808141860 con 0x7f28080ceee0 2023-06-28T20:08:08.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.769+0000 7f280f275700 1 -- 172.21.15.43:0/1285326320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28080728b0 con 0x7f28080ceee0 2023-06-28T20:08:08.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.769+0000 7f27fdffb700 1 -- 172.21.15.43:0/1285326320 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2804018360 con 0x7f28080ceee0 2023-06-28T20:08:08.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.769+0000 7f27fdffb700 1 --2- 172.21.15.43:0/1285326320 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f27ec06ace0 0x7f27ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:08.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.770+0000 7f27fdffb700 1 -- 172.21.15.43:0/1285326320 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f2804089460 con 0x7f28080ceee0 2023-06-28T20:08:08.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.770+0000 7f280d011700 1 --2- 172.21.15.43:0/1285326320 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f27ec06ace0 0x7f27ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:08.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.770+0000 7f280d011700 1 --2- 172.21.15.43:0/1285326320 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f27ec06ace0 0x7f27ec06d190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f27f8006fd0 tx=0x7f27f80090e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:08.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:08.774+0000 7f27fdffb700 1 -- 172.21.15.43:0/1285326320 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2804054480 con 0x7f28080ceee0 2023-06-28T20:08:09.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.026+0000 7f280f275700 1 -- 172.21.15.43:0/1285326320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f280806ef40 con 0x7f28080ceee0 2023-06-28T20:08:09.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.614+0000 7f27fdffb700 1 -- 172.21.15.43:0/1285326320 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2804057aa0 con 0x7f28080ceee0 2023-06-28T20:08:09.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.617+0000 7f280f275700 1 -- 172.21.15.43:0/1285326320 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f27ec06ace0 msgr2=0x7f27ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:09.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.617+0000 7f280f275700 1 --2- 172.21.15.43:0/1285326320 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f27ec06ace0 0x7f27ec06d190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f27f8006fd0 tx=0x7f27f80090e0 comp rx=0 tx=0).stop 2023-06-28T20:08:09.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.617+0000 7f280f275700 1 -- 172.21.15.43:0/1285326320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28080ceee0 msgr2=0x7f28080c47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:09.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.617+0000 7f280f275700 1 --2- 172.21.15.43:0/1285326320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28080ceee0 0x7f28080c47a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f280400c7f0 tx=0x7f280400cb00 comp rx=0 tx=0).stop 2023-06-28T20:08:09.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.618+0000 7f280f275700 1 -- 172.21.15.43:0/1285326320 shutdown_connections 2023-06-28T20:08:09.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.618+0000 7f280f275700 1 --2- 172.21.15.43:0/1285326320 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f27ec06ace0 0x7f27ec06d190 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:09.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.618+0000 7f280f275700 1 --2- 172.21.15.43:0/1285326320 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f280815a5f0 0x7f2808143800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:09.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.618+0000 7f280f275700 1 --2- 172.21.15.43:0/1285326320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f28080afdf0 0x7f28080b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:09.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.618+0000 7f280f275700 1 --2- 172.21.15.43:0/1285326320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28080ceee0 0x7f28080c47a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:09.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.618+0000 7f280f275700 1 -- 172.21.15.43:0/1285326320 >> 172.21.15.43:0/1285326320 conn(0x7f28080664b0 msgr2=0x7f28080b62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:09.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.618+0000 7f280f275700 1 -- 172.21.15.43:0/1285326320 shutdown_connections 2023-06-28T20:08:09.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.619+0000 7f280f275700 1 -- 172.21.15.43:0/1285326320 wait complete. 2023-06-28T20:08:09.638 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-06-28T20:08:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.809+0000 7f3665468700 1 Processor -- start 2023-06-28T20:08:09.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.809+0000 7f3665468700 1 -- start start 2023-06-28T20:08:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.810+0000 7f3665468700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f36600b4f20 0x7f366015d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.810+0000 7f3665468700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36600d0e20 0x7f36600d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.810+0000 7f3665468700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36600d0990 con 0x7f366015ceb0 2023-06-28T20:08:09.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.810+0000 7f3665468700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36600b8960 con 0x7f36600b4f20 2023-06-28T20:08:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.810+0000 7f3665468700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36600ce770 con 0x7f36600d0e20 2023-06-28T20:08:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.810+0000 7f365e7fc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f36600b4f20 0x7f366015d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.811+0000 7f365effd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f366015ceb0 0x7f36600cef70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43826/0 (socket says 172.21.15.43:43826) 2023-06-28T20:08:09.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.811+0000 7f365effd700 1 -- 172.21.15.43:0/539708007 learned_addr learned my addr 172.21.15.43:0/539708007 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.811+0000 7f365f7fe700 1 --2- 172.21.15.43:0/539708007 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36600d0e20 0x7f36600d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.811+0000 7f365e7fc700 1 -- 172.21.15.43:0/539708007 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36600d0e20 msgr2=0x7f36600d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.811+0000 7f365e7fc700 1 --2- 172.21.15.43:0/539708007 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36600d0e20 0x7f36600d12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.811+0000 7f365e7fc700 1 -- 172.21.15.43:0/539708007 >> v1:172.21.15.43:6789/0 conn(0x7f366015ceb0 legacy=0x7f36600cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.811+0000 7f365dffb700 1 -- 172.21.15.43:0/539708007 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 111740641 0 0) 0x7f36600d0990 con 0x7f366015ceb0 2023-06-28T20:08:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.811+0000 7f365e7fc700 1 -- 172.21.15.43:0/539708007 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366015d990 con 0x7f36600b4f20 2023-06-28T20:08:09.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.811+0000 7f365f7fe700 1 --2- 172.21.15.43:0/539708007 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36600d0e20 0x7f36600d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.811+0000 7f365e7fc700 1 --2- 172.21.15.43:0/539708007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36600b4f20 0x7f366015d280 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3648009840 tx=0x7f3648009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db3beadf08a8531 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.811+0000 7f365dffb700 1 -- 172.21.15.43:0/539708007 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648011830 con 0x7f36600b4f20 2023-06-28T20:08:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.811+0000 7f365dffb700 1 -- 172.21.15.43:0/539708007 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3648011990 con 0x7f36600b4f20 2023-06-28T20:08:09.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.811+0000 7f365dffb700 1 -- 172.21.15.43:0/539708007 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648011d20 con 0x7f36600b4f20 2023-06-28T20:08:09.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.812+0000 7f3665468700 1 -- 172.21.15.43:0/539708007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36600b4f20 msgr2=0x7f366015d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:09.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.812+0000 7f3665468700 1 --2- 172.21.15.43:0/539708007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36600b4f20 0x7f366015d280 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3648009840 tx=0x7f3648009b50 comp rx=0 tx=0).stop 2023-06-28T20:08:09.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.812+0000 7f3665468700 1 -- 172.21.15.43:0/539708007 shutdown_connections 2023-06-28T20:08:09.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.812+0000 7f3665468700 1 --2- 172.21.15.43:0/539708007 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36600d0e20 0x7f36600d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:09.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.812+0000 7f3665468700 1 --2- 172.21.15.43:0/539708007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36600b4f20 0x7f366015d280 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:09.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.812+0000 7f3665468700 1 -- 172.21.15.43:0/539708007 >> 172.21.15.43:0/539708007 conn(0x7f366006a5c0 msgr2=0x7f366006a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:09.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.812+0000 7f3665468700 1 -- 172.21.15.43:0/539708007 shutdown_connections 2023-06-28T20:08:09.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.812+0000 7f3665468700 1 -- 172.21.15.43:0/539708007 wait complete. 2023-06-28T20:08:09.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.813+0000 7f3665468700 1 Processor -- start 2023-06-28T20:08:09.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.813+0000 7f3665468700 1 -- start start 2023-06-28T20:08:09.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.814+0000 7f3665468700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36600b4f20 0x7f366014b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:09.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.814+0000 7f3665468700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36600d0e20 0x7f36600a9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:09.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.814+0000 7f3665468700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f366015ceb0 0x7f36600a9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:09.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.814+0000 7f3665468700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36600aae70 con 0x7f36600b4f20 2023-06-28T20:08:09.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.814+0000 7f3665468700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3660151cd0 con 0x7f366015ceb0 2023-06-28T20:08:09.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.814+0000 7f3665468700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36600aa2f0 con 0x7f36600d0e20 2023-06-28T20:08:09.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.814+0000 7f365effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36600b4f20 0x7f366014b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:09.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.814+0000 7f365effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36600b4f20 0x7f366014b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54646/0 (socket says 172.21.15.43:54646) 2023-06-28T20:08:09.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.814+0000 7f365effd700 1 -- 172.21.15.43:0/1642543517 learned_addr learned my addr 172.21.15.43:0/1642543517 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:09.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.814+0000 7f365e7fc700 1 --2- 172.21.15.43:0/1642543517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36600d0e20 0x7f36600a9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:09.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.814+0000 7f365f7fe700 1 --2- 172.21.15.43:0/1642543517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f366015ceb0 0x7f36600a9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:09.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.814+0000 7f365effd700 1 -- 172.21.15.43:0/1642543517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36600d0e20 msgr2=0x7f36600a9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.814+0000 7f365effd700 1 --2- 172.21.15.43:0/1642543517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36600d0e20 0x7f36600a9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.814+0000 7f365effd700 1 -- 172.21.15.43:0/1642543517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f366015ceb0 msgr2=0x7f36600a9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.814+0000 7f365effd700 1 --2- 172.21.15.43:0/1642543517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f366015ceb0 0x7f36600a9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.814+0000 7f365effd700 1 -- 172.21.15.43:0/1642543517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36480094a0 con 0x7f36600b4f20 2023-06-28T20:08:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.814+0000 7f365e7fc700 1 --2- 172.21.15.43:0/1642543517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36600d0e20 0x7f36600a9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:09.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.814+0000 7f365effd700 1 --2- 172.21.15.43:0/1642543517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36600b4f20 0x7f366014b150 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f36500011e0 tx=0x7f365000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.814+0000 7f36577fe700 1 -- 172.21.15.43:0/1642543517 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3650004c80 con 0x7f36600b4f20 2023-06-28T20:08:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.815+0000 7f36577fe700 1 -- 172.21.15.43:0/1642543517 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3650004e00 con 0x7f36600b4f20 2023-06-28T20:08:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.815+0000 7f3665468700 1 -- 172.21.15.43:0/1642543517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36600aa5d0 con 0x7f36600b4f20 2023-06-28T20:08:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.815+0000 7f36577fe700 1 -- 172.21.15.43:0/1642543517 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3650007310 con 0x7f36600b4f20 2023-06-28T20:08:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.815+0000 7f3665468700 1 -- 172.21.15.43:0/1642543517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36600aab90 con 0x7f36600b4f20 2023-06-28T20:08:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.816+0000 7f3665468700 1 -- 172.21.15.43:0/1642543517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36600728b0 con 0x7f36600b4f20 2023-06-28T20:08:09.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.816+0000 7f36577fe700 1 -- 172.21.15.43:0/1642543517 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3650007470 con 0x7f36600b4f20 2023-06-28T20:08:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.816+0000 7f36577fe700 1 --2- 172.21.15.43:0/1642543517 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f364006ace0 0x7f364006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.817+0000 7f36577fe700 1 -- 172.21.15.43:0/1642543517 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f3650089350 con 0x7f36600b4f20 2023-06-28T20:08:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.817+0000 7f365e7fc700 1 --2- 172.21.15.43:0/1642543517 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f364006ace0 0x7f364006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.817+0000 7f365e7fc700 1 --2- 172.21.15.43:0/1642543517 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f364006ace0 0x7f364006d190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f36600c6620 tx=0x7f3648008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:09.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:09.821+0000 7f36577fe700 1 -- 172.21.15.43:0/1642543517 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f36500542d0 con 0x7f36600b4f20 2023-06-28T20:08:10.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.024+0000 7f3665468700 1 -- 172.21.15.43:0/1642543517 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f366015e100 con 0x7f36600b4f20 2023-06-28T20:08:10.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.621+0000 7f36577fe700 1 -- 172.21.15.43:0/1642543517 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7f36500578f0 con 0x7f36600b4f20 2023-06-28T20:08:10.631 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.627+0000 7f3665468700 1 -- 172.21.15.43:0/1642543517 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f364006ace0 msgr2=0x7f364006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:10.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.627+0000 7f3665468700 1 --2- 172.21.15.43:0/1642543517 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f364006ace0 0x7f364006d190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f36600c6620 tx=0x7f3648008030 comp rx=0 tx=0).stop 2023-06-28T20:08:10.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.627+0000 7f3665468700 1 -- 172.21.15.43:0/1642543517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36600b4f20 msgr2=0x7f366014b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:10.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.627+0000 7f3665468700 1 --2- 172.21.15.43:0/1642543517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36600b4f20 0x7f366014b150 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f36500011e0 tx=0x7f365000cae0 comp rx=0 tx=0).stop 2023-06-28T20:08:10.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.627+0000 7f3665468700 1 -- 172.21.15.43:0/1642543517 shutdown_connections 2023-06-28T20:08:10.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.627+0000 7f3665468700 1 --2- 172.21.15.43:0/1642543517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f366015ceb0 0x7f36600a9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:10.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.627+0000 7f3665468700 1 --2- 172.21.15.43:0/1642543517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36600b4f20 0x7f366014b150 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:10.632 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.627+0000 7f3665468700 1 --2- 172.21.15.43:0/1642543517 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f364006ace0 0x7f364006d190 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:10.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.627+0000 7f3665468700 1 --2- 172.21.15.43:0/1642543517 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f36600d0e20 0x7f36600a9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:10.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.627+0000 7f3665468700 1 -- 172.21.15.43:0/1642543517 >> 172.21.15.43:0/1642543517 conn(0x7f366006a5c0 msgr2=0x7f36600d1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:10.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.628+0000 7f3665468700 1 -- 172.21.15.43:0/1642543517 shutdown_connections 2023-06-28T20:08:10.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.628+0000 7f3665468700 1 -- 172.21.15.43:0/1642543517 wait complete. 2023-06-28T20:08:10.633 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2023-06-28T20:08:10.648 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-06-28T20:08:10.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.779+0000 7f4781cbf700 1 Processor -- start 2023-06-28T20:08:10.784 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.779+0000 7f4781cbf700 1 -- start start 2023-06-28T20:08:10.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.780+0000 7f4781cbf700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c06da60 0x7f477c160c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:10.792 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.780+0000 7f4781cbf700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f477c06de30 0x7f477c161160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:10.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.780+0000 7f4781cbf700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f477c0cfa10 con 0x7f477c0af900 2023-06-28T20:08:10.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.780+0000 7f4781cbf700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f477c0d0620 con 0x7f477c06de30 2023-06-28T20:08:10.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.780+0000 7f4781cbf700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477c1616a0 con 0x7f477c06da60 2023-06-28T20:08:10.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.780+0000 7f477affd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f477c0af900 0x7f477c0b2610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43840/0 (socket says 172.21.15.43:43840) 2023-06-28T20:08:10.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.780+0000 7f477affd700 1 -- 172.21.15.43:0/3751971211 learned_addr learned my addr 172.21.15.43:0/3751971211 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:10.793 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.781+0000 7f477b7fe700 1 --2- 172.21.15.43:0/3751971211 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c06da60 0x7f477c160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:10.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.781+0000 7f477bfff700 1 --2- 172.21.15.43:0/3751971211 >> v2:172.21.15.178:3300/0 conn(0x7f477c06de30 0x7f477c161160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:10.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.781+0000 7f477a7fc700 1 -- 172.21.15.43:0/3751971211 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 197718708 0 0) 0x7f477c0cfa10 con 0x7f477c0af900 2023-06-28T20:08:10.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.781+0000 7f477a7fc700 1 -- 172.21.15.43:0/3751971211 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47680035b0 con 0x7f477c0af900 2023-06-28T20:08:10.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.781+0000 7f477a7fc700 1 -- 172.21.15.43:0/3751971211 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1906812611 0 0) 0x7f47680035b0 con 0x7f477c0af900 2023-06-28T20:08:10.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.781+0000 7f477a7fc700 1 -- 172.21.15.43:0/3751971211 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c06da60 msgr2=0x7f477c160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:10.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.781+0000 7f477a7fc700 1 --2- 172.21.15.43:0/3751971211 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c06da60 0x7f477c160c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:10.794 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.781+0000 7f477a7fc700 1 -- 172.21.15.43:0/3751971211 >> v2:172.21.15.178:3300/0 conn(0x7f477c06de30 msgr2=0x7f477c161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:10.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.781+0000 7f477a7fc700 1 --2- 172.21.15.43:0/3751971211 >> v2:172.21.15.178:3300/0 conn(0x7f477c06de30 0x7f477c161160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:10.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.781+0000 7f477a7fc700 1 -- 172.21.15.43:0/3751971211 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c0b03a0 con 0x7f477c0af900 2023-06-28T20:08:10.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.781+0000 7f477a7fc700 1 -- 172.21.15.43:0/3751971211 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f4764001670 con 0x7f477c0af900 2023-06-28T20:08:10.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.781+0000 7f477a7fc700 1 -- 172.21.15.43:0/3751971211 >> v1:172.21.15.43:6789/0 conn(0x7f477c0af900 legacy=0x7f477c0b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:10.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.781+0000 7f477a7fc700 1 --2- 172.21.15.43:0/3751971211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4768003a00 0x7f4768003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:10.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.781+0000 7f477a7fc700 1 --2- 172.21.15.43:0/3751971211 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c06da60 0x7f4768004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:10.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.781+0000 7f477a7fc700 1 --2- 172.21.15.43:0/3751971211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4768004d90 0x7f4768005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:10.795 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.781+0000 7f477a7fc700 1 -- 172.21.15.43:0/3751971211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c0d0620 con 0x7f4768004d90 2023-06-28T20:08:10.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.781+0000 7f477a7fc700 1 -- 172.21.15.43:0/3751971211 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c1616a0 con 0x7f4768003a00 2023-06-28T20:08:10.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.781+0000 7f477a7fc700 1 -- 172.21.15.43:0/3751971211 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476800ba10 con 0x7f477c06da60 2023-06-28T20:08:10.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.781+0000 7f477bfff700 1 --2- 172.21.15.43:0/3751971211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4768004d90 0x7f4768005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:10.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.781+0000 7f477b7fe700 1 --2- 172.21.15.43:0/3751971211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4768003a00 0x7f4768003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:10.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.781+0000 7f477affd700 1 --2- 172.21.15.43:0/3751971211 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c06da60 0x7f4768004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:10.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.782+0000 7f477bfff700 1 -- 172.21.15.43:0/3751971211 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c06da60 msgr2=0x7f4768004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:10.796 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.782+0000 7f477bfff700 1 --2- 172.21.15.43:0/3751971211 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c06da60 0x7f4768004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:10.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.782+0000 7f477bfff700 1 -- 172.21.15.43:0/3751971211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4768003a00 msgr2=0x7f4768003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:10.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.782+0000 7f477bfff700 1 --2- 172.21.15.43:0/3751971211 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4768003a00 0x7f4768003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:10.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.782+0000 7f477bfff700 1 -- 172.21.15.43:0/3751971211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47680035b0 con 0x7f4768004d90 2023-06-28T20:08:10.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.782+0000 7f477bfff700 1 --2- 172.21.15.43:0/3751971211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4768004d90 0x7f4768005200 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4774009fc0 tx=0x7f4768005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4808955636e77d58 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:10.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.782+0000 7f477a7fc700 1 -- 172.21.15.43:0/3751971211 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477400ebd0 con 0x7f4768004d90 2023-06-28T20:08:10.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.782+0000 7f477a7fc700 1 -- 172.21.15.43:0/3751971211 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477400f040 con 0x7f4768004d90 2023-06-28T20:08:10.797 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.782+0000 7f477a7fc700 1 -- 172.21.15.43:0/3751971211 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4774014480 con 0x7f4768004d90 2023-06-28T20:08:10.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.782+0000 7f4781cbf700 1 -- 172.21.15.43:0/3751971211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4768004d90 msgr2=0x7f4768005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:10.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.782+0000 7f4781cbf700 1 --2- 172.21.15.43:0/3751971211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4768004d90 0x7f4768005200 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4774009fc0 tx=0x7f4768005880 comp rx=0 tx=0).stop 2023-06-28T20:08:10.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.782+0000 7f477bfff700 1 -- 172.21.15.43:0/3751971211 reap_dead start 2023-06-28T20:08:10.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.782+0000 7f4781cbf700 1 -- 172.21.15.43:0/3751971211 shutdown_connections 2023-06-28T20:08:10.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.782+0000 7f4781cbf700 1 -- 172.21.15.43:0/3751971211 >> 172.21.15.43:0/3751971211 conn(0x7f477c099f30 msgr2=0x7f477c0d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:10.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.783+0000 7f4781cbf700 1 -- 172.21.15.43:0/3751971211 shutdown_connections 2023-06-28T20:08:10.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.783+0000 7f4781cbf700 1 -- 172.21.15.43:0/3751971211 wait complete. 2023-06-28T20:08:10.798 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.784+0000 7f4781cbf700 1 Processor -- start 2023-06-28T20:08:10.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.784+0000 7f4781cbf700 1 -- start start 2023-06-28T20:08:10.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.785+0000 7f4781cbf700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c159890 0x7f477c15f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:10.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.785+0000 7f4781cbf700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477c160d20 0x7f477c1f2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:10.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.785+0000 7f4781cbf700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f477c0c4fe0 0x7f477c0c5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:10.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.785+0000 7f4781cbf700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c0c59a0 con 0x7f477c160d20 2023-06-28T20:08:10.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.785+0000 7f4781cbf700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c1f33c0 con 0x7f477c0c4fe0 2023-06-28T20:08:10.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.785+0000 7f4781cbf700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477c159c60 con 0x7f477c159890 2023-06-28T20:08:10.799 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.785+0000 7f477affd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477c160d20 0x7f477c1f2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:10.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.785+0000 7f477affd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477c160d20 0x7f477c1f2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54668/0 (socket says 172.21.15.43:54668) 2023-06-28T20:08:10.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.785+0000 7f477affd700 1 -- 172.21.15.43:0/1426914508 learned_addr learned my addr 172.21.15.43:0/1426914508 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:10.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.785+0000 7f477bfff700 1 --2- 172.21.15.43:0/1426914508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f477c0c4fe0 0x7f477c0c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:10.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.785+0000 7f477b7fe700 1 --2- 172.21.15.43:0/1426914508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c159890 0x7f477c15f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:10.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.785+0000 7f477affd700 1 -- 172.21.15.43:0/1426914508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c159890 msgr2=0x7f477c15f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:10.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.785+0000 7f477affd700 1 --2- 172.21.15.43:0/1426914508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c159890 0x7f477c15f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:10.800 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.785+0000 7f477affd700 1 -- 172.21.15.43:0/1426914508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f477c0c4fe0 msgr2=0x7f477c0c5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:10.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.785+0000 7f477affd700 1 --2- 172.21.15.43:0/1426914508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f477c0c4fe0 0x7f477c0c5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:10.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.785+0000 7f477affd700 1 -- 172.21.15.43:0/1426914508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c15fc80 con 0x7f477c160d20 2023-06-28T20:08:10.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.785+0000 7f477affd700 1 --2- 172.21.15.43:0/1426914508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477c160d20 0x7f477c1f2e80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4764004fd0 tx=0x7f47640076a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:10.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.785+0000 7f4773fff700 1 -- 172.21.15.43:0/1426914508 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476400f8a0 con 0x7f477c160d20 2023-06-28T20:08:10.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.785+0000 7f4773fff700 1 -- 172.21.15.43:0/1426914508 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476400fa20 con 0x7f477c160d20 2023-06-28T20:08:10.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.785+0000 7f4773fff700 1 -- 172.21.15.43:0/1426914508 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476400fdd0 con 0x7f477c160d20 2023-06-28T20:08:10.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.785+0000 7f4781cbf700 1 -- 172.21.15.43:0/1426914508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477c15ff60 con 0x7f477c160d20 2023-06-28T20:08:10.801 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.785+0000 7f4781cbf700 1 -- 172.21.15.43:0/1426914508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477c160520 con 0x7f477c160d20 2023-06-28T20:08:10.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.786+0000 7f4781cbf700 1 -- 172.21.15.43:0/1426914508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f477c0728b0 con 0x7f477c160d20 2023-06-28T20:08:10.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.787+0000 7f4773fff700 1 -- 172.21.15.43:0/1426914508 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f476401d460 con 0x7f477c160d20 2023-06-28T20:08:10.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.787+0000 7f4773fff700 1 --2- 172.21.15.43:0/1426914508 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f475c06ace0 0x7f475c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:10.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.787+0000 7f4773fff700 1 -- 172.21.15.43:0/1426914508 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5768+0+0 (secure 0 0 0) 0x7f4764089530 con 0x7f477c160d20 2023-06-28T20:08:10.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.787+0000 7f477b7fe700 1 --2- 172.21.15.43:0/1426914508 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f475c06ace0 0x7f475c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:10.802 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.788+0000 7f477b7fe700 1 --2- 172.21.15.43:0/1426914508 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f475c06ace0 0x7f475c06d190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f476c009b00 tx=0x7f476c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:10.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.792+0000 7f4773fff700 1 -- 172.21.15.43:0/1426914508 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f47640547b0 con 0x7f477c160d20 2023-06-28T20:08:10.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:10.993+0000 7f4781cbf700 1 -- 172.21.15.43:0/1426914508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f477c09a3c0 con 0x7f477c160d20 2023-06-28T20:08:11.629 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.625+0000 7f4773fff700 1 -- 172.21.15.43:0/1426914508 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4764007c20 con 0x7f477c160d20 2023-06-28T20:08:11.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.629+0000 7f4781cbf700 1 -- 172.21.15.43:0/1426914508 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f475c06ace0 msgr2=0x7f475c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:11.633 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.629+0000 7f4781cbf700 1 --2- 172.21.15.43:0/1426914508 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f475c06ace0 0x7f475c06d190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f476c009b00 tx=0x7f476c008040 comp rx=0 tx=0).stop 2023-06-28T20:08:11.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.629+0000 7f4781cbf700 1 -- 172.21.15.43:0/1426914508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477c160d20 msgr2=0x7f477c1f2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:11.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.629+0000 7f4781cbf700 1 --2- 172.21.15.43:0/1426914508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477c160d20 0x7f477c1f2e80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4764004fd0 tx=0x7f47640076a0 comp rx=0 tx=0).stop 2023-06-28T20:08:11.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.629+0000 7f4781cbf700 1 -- 172.21.15.43:0/1426914508 shutdown_connections 2023-06-28T20:08:11.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.629+0000 7f4781cbf700 1 --2- 172.21.15.43:0/1426914508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f477c0c4fe0 0x7f477c0c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:11.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.629+0000 7f4781cbf700 1 --2- 172.21.15.43:0/1426914508 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f475c06ace0 0x7f475c06d190 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:11.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.629+0000 7f4781cbf700 1 --2- 172.21.15.43:0/1426914508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f477c159890 0x7f477c15f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:11.634 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.629+0000 7f4781cbf700 1 --2- 172.21.15.43:0/1426914508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477c160d20 0x7f477c1f2e80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:11.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.629+0000 7f4781cbf700 1 -- 172.21.15.43:0/1426914508 >> 172.21.15.43:0/1426914508 conn(0x7f477c099f30 msgr2=0x7f477c15d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:11.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.630+0000 7f4781cbf700 1 -- 172.21.15.43:0/1426914508 shutdown_connections 2023-06-28T20:08:11.635 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.630+0000 7f4781cbf700 1 -- 172.21.15.43:0/1426914508 wait complete. 2023-06-28T20:08:11.636 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2023-06-28T20:08:11.650 INFO:tasks.ceph.mds.a:Restarting daemon 2023-06-28T20:08:11.651 INFO:tasks.ceph.mds.a:Stopping old one... 2023-06-28T20:08:11.651 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-06-28T20:08:11.651 INFO:teuthology.orchestra.run:waiting for 300 2023-06-28T20:08:11.652 INFO:tasks.ceph.mds.c:Restarting daemon 2023-06-28T20:08:11.652 INFO:tasks.ceph.mds.c:Stopping old one... 2023-06-28T20:08:11.652 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-06-28T20:08:11.652 INFO:teuthology.orchestra.run:waiting for 300 2023-06-28T20:08:11.652 INFO:tasks.ceph.mds.b:Restarting daemon 2023-06-28T20:08:11.653 INFO:tasks.ceph.mds.b:Stopping old one... 2023-06-28T20:08:11.653 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-06-28T20:08:11.653 INFO:teuthology.orchestra.run:waiting for 300 2023-06-28T20:08:11.653 INFO:tasks.ceph.mds.d:Restarting daemon 2023-06-28T20:08:11.654 INFO:tasks.ceph.mds.d:Stopping old one... 2023-06-28T20:08:11.654 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-06-28T20:08:11.654 INFO:teuthology.orchestra.run:waiting for 300 2023-06-28T20:08:11.661 INFO:tasks.ceph.mds.a:Stopped 2023-06-28T20:08:11.662 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-06-28T20:08:11.663 INFO:tasks.ceph.mds.c:Stopped 2023-06-28T20:08:11.663 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-06-28T20:08:11.665 INFO:tasks.ceph.mds.b:Stopped 2023-06-28T20:08:11.666 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-06-28T20:08:11.666 INFO:tasks.ceph.mds.d:Stopped 2023-06-28T20:08:11.667 DEBUG:teuthology.orchestra.run.smithi178:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-06-28T20:08:11.704 INFO:tasks.ceph.mds.a:Started 2023-06-28T20:08:11.705 DEBUG:teuthology.parallel:result is None 2023-06-28T20:08:11.705 INFO:tasks.ceph.mds.c:Started 2023-06-28T20:08:11.706 DEBUG:teuthology.parallel:result is None 2023-06-28T20:08:11.707 INFO:tasks.ceph.mds.b:Started 2023-06-28T20:08:11.708 DEBUG:teuthology.parallel:result is None 2023-06-28T20:08:11.708 INFO:tasks.ceph.mds.d:Started 2023-06-28T20:08:11.709 DEBUG:teuthology.parallel:result is None 2023-06-28T20:08:11.709 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-06-28T20:08:11.793 INFO:tasks.ceph.mds.d.smithi178.stdout:starting mds.d at 2023-06-28T20:08:11.793 INFO:tasks.ceph.mds.b.smithi178.stdout:starting mds.b at 2023-06-28T20:08:11.794 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2023-06-28T20:08:11.794 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2023-06-28T20:08:11.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.841+0000 7fb283981700 1 Processor -- start 2023-06-28T20:08:11.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.841+0000 7fb283981700 1 -- start start 2023-06-28T20:08:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.842+0000 7fb283981700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb27c1592d0 0x7fb27c0b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:11.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.842+0000 7fb283981700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb27c1596a0 0x7fb27c0b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:11.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.842+0000 7fb283981700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb27c070b40 con 0x7fb27c071900 2023-06-28T20:08:11.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.842+0000 7fb283981700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb27c070e20 con 0x7fb27c1596a0 2023-06-28T20:08:11.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.842+0000 7fb283981700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c071100 con 0x7fb27c1592d0 2023-06-28T20:08:11.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.842+0000 7fb280f1c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb27c1596a0 0x7fb27c0b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:11.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.842+0000 7fb280f1c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb27c1596a0 0x7fb27c0b1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54700/0 (socket says 172.21.15.43:54700) 2023-06-28T20:08:11.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.842+0000 7fb280f1c700 1 -- 172.21.15.43:0/1732634450 learned_addr learned my addr 172.21.15.43:0/1732634450 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.843+0000 7fb28171d700 1 --2- 172.21.15.43:0/1732634450 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb27c1592d0 0x7fb27c0b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.843+0000 7fb280f1c700 1 -- 172.21.15.43:0/1732634450 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb27c1592d0 msgr2=0x7fb27c0b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.843+0000 7fb280f1c700 1 --2- 172.21.15.43:0/1732634450 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb27c1592d0 0x7fb27c0b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.843+0000 7fb280f1c700 1 -- 172.21.15.43:0/1732634450 >> v1:172.21.15.178:6789/0 conn(0x7fb27c071900 legacy=0x7fb27c15d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.843+0000 7fb280f1c700 1 -- 172.21.15.43:0/1732634450 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27c0713e0 con 0x7fb27c1596a0 2023-06-28T20:08:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.843+0000 7fb280f1c700 1 --2- 172.21.15.43:0/1732634450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb27c1596a0 0x7fb27c0b1f10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb2780098f0 tx=0x7fb278009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1389e0f1b3f731a6 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.843+0000 7fb273fff700 1 -- 172.21.15.43:0/1732634450 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb278004230 con 0x7fb27c1596a0 2023-06-28T20:08:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.843+0000 7fb273fff700 1 -- 172.21.15.43:0/1732634450 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb278004390 con 0x7fb27c1596a0 2023-06-28T20:08:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.843+0000 7fb273fff700 1 -- 172.21.15.43:0/1732634450 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2780036a0 con 0x7fb27c1596a0 2023-06-28T20:08:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.843+0000 7fb283981700 1 -- 172.21.15.43:0/1732634450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb27c1596a0 msgr2=0x7fb27c0b1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.843+0000 7fb283981700 1 --2- 172.21.15.43:0/1732634450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb27c1596a0 0x7fb27c0b1f10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb2780098f0 tx=0x7fb278009c00 comp rx=0 tx=0).stop 2023-06-28T20:08:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.844+0000 7fb283981700 1 -- 172.21.15.43:0/1732634450 shutdown_connections 2023-06-28T20:08:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.844+0000 7fb283981700 1 --2- 172.21.15.43:0/1732634450 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb27c1592d0 0x7fb27c0b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.844+0000 7fb283981700 1 --2- 172.21.15.43:0/1732634450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb27c1596a0 0x7fb27c0b1f10 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.844+0000 7fb283981700 1 -- 172.21.15.43:0/1732634450 >> 172.21.15.43:0/1732634450 conn(0x7fb27c06b000 msgr2=0x7fb27c06b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.844+0000 7fb283981700 1 -- 172.21.15.43:0/1732634450 shutdown_connections 2023-06-28T20:08:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.844+0000 7fb283981700 1 -- 172.21.15.43:0/1732634450 wait complete. 2023-06-28T20:08:11.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.845+0000 7fb283981700 1 Processor -- start 2023-06-28T20:08:11.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.845+0000 7fb283981700 1 -- start start 2023-06-28T20:08:11.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.846+0000 7fb283981700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb27c071900 0x7fb27c15c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.846+0000 7fb283981700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb27c1592d0 0x7fb27c1e4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.846+0000 7fb283981700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb27c1596a0 0x7fb27c1530d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.846+0000 7fb283981700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c15c000 con 0x7fb27c071900 2023-06-28T20:08:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.846+0000 7fb283981700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c15c960 con 0x7fb27c1592d0 2023-06-28T20:08:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.846+0000 7fb283981700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27c1e4540 con 0x7fb27c1596a0 2023-06-28T20:08:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.846+0000 7fb28171d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb27c071900 0x7fb27c15c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.846+0000 7fb28171d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb27c071900 0x7fb27c15c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54706/0 (socket says 172.21.15.43:54706) 2023-06-28T20:08:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.846+0000 7fb28171d700 1 -- 172.21.15.43:0/1179455864 learned_addr learned my addr 172.21.15.43:0/1179455864 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.846+0000 7fb280f1c700 1 --2- 172.21.15.43:0/1179455864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb27c1592d0 0x7fb27c1e4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.846+0000 7fb281f1e700 1 --2- 172.21.15.43:0/1179455864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb27c1596a0 0x7fb27c1530d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.846+0000 7fb28171d700 1 -- 172.21.15.43:0/1179455864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb27c1596a0 msgr2=0x7fb27c1530d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.846+0000 7fb28171d700 1 --2- 172.21.15.43:0/1179455864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb27c1596a0 0x7fb27c1530d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.846+0000 7fb28171d700 1 -- 172.21.15.43:0/1179455864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb27c1592d0 msgr2=0x7fb27c1e4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.846+0000 7fb28171d700 1 --2- 172.21.15.43:0/1179455864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb27c1592d0 0x7fb27c1e4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.846+0000 7fb28171d700 1 -- 172.21.15.43:0/1179455864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb278009550 con 0x7fb27c071900 2023-06-28T20:08:11.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.846+0000 7fb28171d700 1 --2- 172.21.15.43:0/1179455864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb27c071900 0x7fb27c15c420 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00e840 tx=0x7fb26c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:11.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.846+0000 7fb271ffb700 1 -- 172.21.15.43:0/1179455864 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c013610 con 0x7fb27c071900 2023-06-28T20:08:11.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.846+0000 7fb271ffb700 1 -- 172.21.15.43:0/1179455864 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c013770 con 0x7fb27c071900 2023-06-28T20:08:11.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.846+0000 7fb271ffb700 1 -- 172.21.15.43:0/1179455864 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26c013b20 con 0x7fb27c071900 2023-06-28T20:08:11.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.846+0000 7fb283981700 1 -- 172.21.15.43:0/1179455864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb27c15b380 con 0x7fb27c071900 2023-06-28T20:08:11.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.847+0000 7fb283981700 1 -- 172.21.15.43:0/1179455864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb27c15b940 con 0x7fb27c071900 2023-06-28T20:08:11.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.848+0000 7fb283981700 1 -- 172.21.15.43:0/1179455864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb27c0cc3e0 con 0x7fb27c071900 2023-06-28T20:08:11.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.848+0000 7fb271ffb700 1 -- 172.21.15.43:0/1179455864 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb26c013c80 con 0x7fb27c071900 2023-06-28T20:08:11.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.848+0000 7fb271ffb700 1 --2- 172.21.15.43:0/1179455864 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb26006ace0 0x7fb26006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:11.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.849+0000 7fb280f1c700 1 --2- 172.21.15.43:0/1179455864 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb26006ace0 0x7fb26006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:11.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.849+0000 7fb271ffb700 1 -- 172.21.15.43:0/1179455864 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fb26c088e10 con 0x7fb27c071900 2023-06-28T20:08:11.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.849+0000 7fb280f1c700 1 --2- 172.21.15.43:0/1179455864 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb26006ace0 0x7fb26006d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb27c159ab0 tx=0x7fb278008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:11.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:11.853+0000 7fb271ffb700 1 -- 172.21.15.43:0/1179455864 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb26c08c030 con 0x7fb27c071900 2023-06-28T20:08:12.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:12.088+0000 7fb283981700 1 -- 172.21.15.43:0/1179455864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb27c0cf3a0 con 0x7fb27c071900 2023-06-28T20:08:12.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:12.933+0000 7fb271ffb700 1 -- 172.21.15.43:0/1179455864 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v31) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb26c021080 con 0x7fb27c071900 2023-06-28T20:08:12.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:12.936+0000 7fb283981700 1 -- 172.21.15.43:0/1179455864 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb26006ace0 msgr2=0x7fb26006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:12.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:12.936+0000 7fb283981700 1 --2- 172.21.15.43:0/1179455864 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb26006ace0 0x7fb26006d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb27c159ab0 tx=0x7fb278008030 comp rx=0 tx=0).stop 2023-06-28T20:08:12.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:12.936+0000 7fb283981700 1 -- 172.21.15.43:0/1179455864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb27c071900 msgr2=0x7fb27c15c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:12.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:12.936+0000 7fb283981700 1 --2- 172.21.15.43:0/1179455864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb27c071900 0x7fb27c15c420 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00e840 tx=0x7fb26c00ec00 comp rx=0 tx=0).stop 2023-06-28T20:08:12.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:12.936+0000 7fb283981700 1 -- 172.21.15.43:0/1179455864 shutdown_connections 2023-06-28T20:08:12.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:12.936+0000 7fb283981700 1 --2- 172.21.15.43:0/1179455864 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb26006ace0 0x7fb26006d190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:12.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:12.936+0000 7fb283981700 1 --2- 172.21.15.43:0/1179455864 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb27c1596a0 0x7fb27c1530d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:12.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:12.936+0000 7fb283981700 1 --2- 172.21.15.43:0/1179455864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb27c071900 0x7fb27c15c420 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:12.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:12.936+0000 7fb283981700 1 --2- 172.21.15.43:0/1179455864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb27c1592d0 0x7fb27c1e4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:12.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:12.936+0000 7fb283981700 1 -- 172.21.15.43:0/1179455864 >> 172.21.15.43:0/1179455864 conn(0x7fb27c06b000 msgr2=0x7fb27c155dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:12.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:12.937+0000 7fb283981700 1 -- 172.21.15.43:0/1179455864 shutdown_connections 2023-06-28T20:08:12.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:12.937+0000 7fb283981700 1 -- 172.21.15.43:0/1179455864 wait complete. 2023-06-28T20:08:12.945 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2023-06-28T20:08:12.956 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-06-28T20:08:13.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.084+0000 7f6621c13700 1 Processor -- start 2023-06-28T20:08:13.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.084+0000 7f6621c13700 1 -- start start 2023-06-28T20:08:13.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.085+0000 7f6621c13700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f661c0af060 0x7f661c0cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:13.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.085+0000 7f6621c13700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f661c06ca80 0x7f661c069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:13.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.085+0000 7f6621c13700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661c071410 con 0x7f661c06ce80 2023-06-28T20:08:13.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.085+0000 7f6621c13700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f661c0716b0 con 0x7f661c0af060 2023-06-28T20:08:13.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.085+0000 7f6621c13700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661c071990 con 0x7f661c06ca80 2023-06-28T20:08:13.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.086+0000 7f661affd700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f661c06ca80 0x7f661c069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:13.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.086+0000 7f661b7fe700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f661c0af060 0x7f661c0cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:13.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.086+0000 7f661bfff700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f661c06ce80 0x7f661c0ac870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:43032/0 (socket says 172.21.15.43:43032) 2023-06-28T20:08:13.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.086+0000 7f661bfff700 1 -- 172.21.15.43:0/3293010017 learned_addr learned my addr 172.21.15.43:0/3293010017 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:13.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.086+0000 7f661affd700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f661c06ca80 0x7f661c069ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.43:38692/0 (socket says 172.21.15.43:38692) 2023-06-28T20:08:13.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.086+0000 7f661b7fe700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f661c0af060 0x7f661c0cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:40782/0 (socket says 172.21.15.43:40782) 2023-06-28T20:08:13.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.086+0000 7f661a7fc700 1 -- 172.21.15.43:0/3293010017 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251680609 0 0) 0x7f661c071410 con 0x7f661c06ce80 2023-06-28T20:08:13.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.086+0000 7f661a7fc700 1 -- 172.21.15.43:0/3293010017 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66080035b0 con 0x7f661c06ce80 2023-06-28T20:08:13.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.086+0000 7f661affd700 1 -- 172.21.15.43:0/3293010017 >> v2:172.21.15.178:3300/0 conn(0x7f661c0af060 msgr2=0x7f661c0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:13.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.086+0000 7f661affd700 1 --2- 172.21.15.43:0/3293010017 >> v2:172.21.15.178:3300/0 conn(0x7f661c0af060 0x7f661c0cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.086+0000 7f661affd700 1 -- 172.21.15.43:0/3293010017 >> v1:172.21.15.178:6789/0 conn(0x7f661c06ce80 legacy=0x7f661c0ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:13.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.086+0000 7f661affd700 1 -- 172.21.15.43:0/3293010017 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661c0af430 con 0x7f661c06ca80 2023-06-28T20:08:13.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.086+0000 7f661b7fe700 1 --2- 172.21.15.43:0/3293010017 >> v2:172.21.15.178:3300/0 conn(0x7f661c0af060 0x7f661c0cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:08:13.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.086+0000 7f661affd700 1 --2- 172.21.15.43:0/3293010017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f661c06ca80 0x7f661c069ef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6604009840 tx=0x7f6604009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=854395151ad52b4f server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:13.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.086+0000 7f661a7fc700 1 -- 172.21.15.43:0/3293010017 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6604011830 con 0x7f661c06ca80 2023-06-28T20:08:13.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.086+0000 7f661a7fc700 1 -- 172.21.15.43:0/3293010017 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6604011990 con 0x7f661c06ca80 2023-06-28T20:08:13.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.086+0000 7f661a7fc700 1 -- 172.21.15.43:0/3293010017 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6604011d20 con 0x7f661c06ca80 2023-06-28T20:08:13.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.087+0000 7f6621c13700 1 -- 172.21.15.43:0/3293010017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f661c06ca80 msgr2=0x7f661c069ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:13.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.087+0000 7f6621c13700 1 --2- 172.21.15.43:0/3293010017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f661c06ca80 0x7f661c069ef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6604009840 tx=0x7f6604009b50 comp rx=0 tx=0).stop 2023-06-28T20:08:13.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.087+0000 7f6621c13700 1 -- 172.21.15.43:0/3293010017 shutdown_connections 2023-06-28T20:08:13.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.087+0000 7f6621c13700 1 --2- 172.21.15.43:0/3293010017 >> v2:172.21.15.178:3300/0 conn(0x7f661c0af060 0x7f661c0cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.087+0000 7f6621c13700 1 --2- 172.21.15.43:0/3293010017 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f661c06ca80 0x7f661c069ef0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.087+0000 7f6621c13700 1 -- 172.21.15.43:0/3293010017 >> 172.21.15.43:0/3293010017 conn(0x7f661c0b1720 msgr2=0x7f661c0b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:13.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.087+0000 7f6621c13700 1 -- 172.21.15.43:0/3293010017 shutdown_connections 2023-06-28T20:08:13.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.087+0000 7f6621c13700 1 -- 172.21.15.43:0/3293010017 wait complete. 2023-06-28T20:08:13.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.088+0000 7f6621c13700 1 Processor -- start 2023-06-28T20:08:13.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.088+0000 7f6621c13700 1 -- start start 2023-06-28T20:08:13.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.089+0000 7f6621c13700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f661c06ce80 0x7f661c15f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:13.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.089+0000 7f6621c13700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f661c0af060 0x7f661c159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:13.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.089+0000 7f6621c13700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661c0cbc40 0x7f661c0c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:13.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.089+0000 7f6621c13700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661c15ca20 con 0x7f661c06ce80 2023-06-28T20:08:13.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.089+0000 7f6621c13700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661c159760 con 0x7f661c0cbc40 2023-06-28T20:08:13.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.089+0000 7f6621c13700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661c0cc470 con 0x7f661c0af060 2023-06-28T20:08:13.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.089+0000 7f661b7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f661c06ce80 0x7f661c15f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:13.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.089+0000 7f661b7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f661c06ce80 0x7f661c15f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54724/0 (socket says 172.21.15.43:54724) 2023-06-28T20:08:13.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.089+0000 7f661b7fe700 1 -- 172.21.15.43:0/992472304 learned_addr learned my addr 172.21.15.43:0/992472304 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:13.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.090+0000 7f661affd700 1 --2- 172.21.15.43:0/992472304 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f661c0af060 0x7f661c159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:13.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.090+0000 7f661bfff700 1 --2- 172.21.15.43:0/992472304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661c0cbc40 0x7f661c0c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:13.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.090+0000 7f661b7fe700 1 -- 172.21.15.43:0/992472304 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f661c0af060 msgr2=0x7f661c159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:13.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.090+0000 7f661b7fe700 1 --2- 172.21.15.43:0/992472304 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f661c0af060 0x7f661c159220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.090+0000 7f661b7fe700 1 -- 172.21.15.43:0/992472304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661c0cbc40 msgr2=0x7f661c0c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:13.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.090+0000 7f661b7fe700 1 --2- 172.21.15.43:0/992472304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661c0cbc40 0x7f661c0c5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.090+0000 7f661b7fe700 1 -- 172.21.15.43:0/992472304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f660c0094a0 con 0x7f661c06ce80 2023-06-28T20:08:13.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.090+0000 7f661b7fe700 1 --2- 172.21.15.43:0/992472304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f661c06ce80 0x7f661c15f4c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f660c00ebe0 tx=0x7f660c009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:13.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.090+0000 7f6613fff700 1 -- 172.21.15.43:0/992472304 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c00c8b0 con 0x7f661c06ce80 2023-06-28T20:08:13.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.090+0000 7f6613fff700 1 -- 172.21.15.43:0/992472304 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660c00ca10 con 0x7f661c06ce80 2023-06-28T20:08:13.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.090+0000 7f6621c13700 1 -- 172.21.15.43:0/992472304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66040094a0 con 0x7f661c06ce80 2023-06-28T20:08:13.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.090+0000 7f6613fff700 1 -- 172.21.15.43:0/992472304 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c00cda0 con 0x7f661c06ce80 2023-06-28T20:08:13.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.090+0000 7f6621c13700 1 -- 172.21.15.43:0/992472304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f661c0cca30 con 0x7f661c06ce80 2023-06-28T20:08:13.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.091+0000 7f6621c13700 1 -- 172.21.15.43:0/992472304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f661c0728b0 con 0x7f661c06ce80 2023-06-28T20:08:13.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.091+0000 7f6613fff700 1 -- 172.21.15.43:0/992472304 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f660c013470 con 0x7f661c06ce80 2023-06-28T20:08:13.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.092+0000 7f6613fff700 1 --2- 172.21.15.43:0/992472304 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f65fc06ace0 0x7f65fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:13.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.092+0000 7f6613fff700 1 -- 172.21.15.43:0/992472304 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f660c089250 con 0x7f661c06ce80 2023-06-28T20:08:13.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.092+0000 7f661affd700 1 --2- 172.21.15.43:0/992472304 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f65fc06ace0 0x7f65fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:13.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.092+0000 7f661affd700 1 --2- 172.21.15.43:0/992472304 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f65fc06ace0 0x7f65fc06d190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f661c0ae670 tx=0x7f6604008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:13.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.096+0000 7f6613fff700 1 -- 172.21.15.43:0/992472304 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f660c0546e0 con 0x7f661c06ce80 2023-06-28T20:08:13.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.374+0000 7f6621c13700 1 -- 172.21.15.43:0/992472304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f661c0ca040 con 0x7f661c06ce80 2023-06-28T20:08:13.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.374+0000 7f6613fff700 1 -- 172.21.15.43:0/992472304 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f660c057d00 con 0x7f661c06ce80 2023-06-28T20:08:13.379 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:13.379 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-06-28T20:08:13.379 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2023-06-28T20:08:13.379 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.379 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2023-06-28T20:08:13.379 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDCkpxk3XrHOBAAHldrZ+MiPeCvaRDv/Tgx+A==", 2023-06-28T20:08:13.380 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.380 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-06-28T20:08:13.380 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-06-28T20:08:13.380 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-06-28T20:08:13.380 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-06-28T20:08:13.380 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.380 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.381 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.381 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2023-06-28T20:08:13.381 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDDkpxkH7KzCRAAQINxW1rrA8FdxRr3sU0Vjg==", 2023-06-28T20:08:13.381 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.382 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-06-28T20:08:13.382 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-06-28T20:08:13.382 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-06-28T20:08:13.382 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-06-28T20:08:13.382 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.382 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.382 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.383 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2023-06-28T20:08:13.383 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDDkpxkScSiBBAAUerAFKjwrjArDAUJ/R9DMg==", 2023-06-28T20:08:13.383 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.383 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-06-28T20:08:13.383 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-06-28T20:08:13.383 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-06-28T20:08:13.383 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-06-28T20:08:13.383 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.384 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.384 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.384 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2023-06-28T20:08:13.384 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDDkpxkQ6oIERAAbTYaJNOpl9DlB1mx8KMm/w==", 2023-06-28T20:08:13.384 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.384 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-06-28T20:08:13.385 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-06-28T20:08:13.385 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-06-28T20:08:13.385 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-06-28T20:08:13.385 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.385 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.386 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.386 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2023-06-28T20:08:13.386 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDFkpxkpy8vABAA96KfRPxwK8A6BhwkhzpWSA==", 2023-06-28T20:08:13.386 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.386 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-06-28T20:08:13.386 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-06-28T20:08:13.386 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-06-28T20:08:13.387 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.387 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.387 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.387 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2023-06-28T20:08:13.387 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDIkpxkigYbARAAlX+rtKMQdqlHkBjs5qQllg==", 2023-06-28T20:08:13.387 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.387 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-06-28T20:08:13.388 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-06-28T20:08:13.388 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-06-28T20:08:13.388 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.388 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.388 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2023-06-28T20:08:13.388 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDLkpxkMJgUAhAAo/lTCTzRKdBb9gkiI5vavQ==", 2023-06-28T20:08:13.388 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.389 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-06-28T20:08:13.389 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-06-28T20:08:13.389 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-06-28T20:08:13.389 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.389 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.389 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2023-06-28T20:08:13.390 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDOkpxkNbi3AhAABEA96FPfDzl8dRSfBMzv8w==", 2023-06-28T20:08:13.390 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.390 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-06-28T20:08:13.390 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-06-28T20:08:13.390 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-06-28T20:08:13.390 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.391 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.391 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2023-06-28T20:08:13.391 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDSkpxkeYL6DRAA8m/izs8fTALY+tCkH39GGg==", 2023-06-28T20:08:13.391 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.391 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-06-28T20:08:13.391 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-06-28T20:08:13.391 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-06-28T20:08:13.391 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.392 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.392 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2023-06-28T20:08:13.392 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDVkpxkl+q7DBAAhXlNHW3APq+oCYnzyovY2A==", 2023-06-28T20:08:13.392 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.392 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-06-28T20:08:13.392 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-06-28T20:08:13.393 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-06-28T20:08:13.393 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.393 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.393 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2023-06-28T20:08:13.393 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDYkpxkUHFoDBAAvV2PKXUosUjwLnzo0bT0zw==", 2023-06-28T20:08:13.393 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.393 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-06-28T20:08:13.394 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-06-28T20:08:13.394 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-06-28T20:08:13.394 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.394 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.394 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2023-06-28T20:08:13.394 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDbkpxkLYptCxAA8egRdpdmWEozABFeQSao4g==", 2023-06-28T20:08:13.394 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.395 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2023-06-28T20:08:13.395 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2023-06-28T20:08:13.395 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-06-28T20:08:13.395 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.395 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2023-06-28T20:08:13.396 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDDkpxkmceMFRAAjFOucYVKNehSzr5v2Q0vbg==", 2023-06-28T20:08:13.396 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.396 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-06-28T20:08:13.396 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-06-28T20:08:13.396 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-06-28T20:08:13.396 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-06-28T20:08:13.396 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.397 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.397 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.397 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2023-06-28T20:08:13.397 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDDkpxkULRaHBAAU3pR+3uFPsVd1kDHh37shg==", 2023-06-28T20:08:13.397 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.397 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-06-28T20:08:13.397 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-06-28T20:08:13.397 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-06-28T20:08:13.398 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-06-28T20:08:13.398 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.398 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.398 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.398 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2023-06-28T20:08:13.398 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDDkpxkqMP8GBAAF36pZfz6787IM6vFDAGIqA==", 2023-06-28T20:08:13.398 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.399 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-06-28T20:08:13.399 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-06-28T20:08:13.399 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-06-28T20:08:13.399 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-06-28T20:08:13.399 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.399 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.399 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.399 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2023-06-28T20:08:13.400 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDDkpxksb4fIhAASbu90RGfkIMaS2tD7RpFRw==", 2023-06-28T20:08:13.400 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.400 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2023-06-28T20:08:13.400 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2023-06-28T20:08:13.400 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2023-06-28T20:08:13.400 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2023-06-28T20:08:13.400 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.401 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.401 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.401 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2023-06-28T20:08:13.401 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDCkpxkWn3WGRAA1UTEIGDY9seaMUrwboQ4lw==", 2023-06-28T20:08:13.401 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.401 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-06-28T20:08:13.401 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2023-06-28T20:08:13.401 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2023-06-28T20:08:13.402 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-06-28T20:08:13.402 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.402 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.402 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.402 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-mds", 2023-06-28T20:08:13.402 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDmkpxkbKgBChAAuKRHE26Ltc+tUj7jsCmSaA==", 2023-06-28T20:08:13.402 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.402 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-mds" 2023-06-28T20:08:13.403 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.403 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.403 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.403 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-mgr", 2023-06-28T20:08:13.403 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDmkpxk47UBChAA4XP1gzrCgaCKGNyFM4oSEg==", 2023-06-28T20:08:13.403 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.403 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-mgr" 2023-06-28T20:08:13.404 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.404 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.404 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-osd", 2023-06-28T20:08:13.404 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDmkpxkOcABChAAqiiids8GcjOd+UgE44Uq0w==", 2023-06-28T20:08:13.404 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.404 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-osd" 2023-06-28T20:08:13.404 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.405 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.405 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.405 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-rbd", 2023-06-28T20:08:13.405 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDmkpxkTssBChAAYBtltkcVdx/YYmUrA/8hEA==", 2023-06-28T20:08:13.405 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.405 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-rbd" 2023-06-28T20:08:13.405 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.406 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.406 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-06-28T20:08:13.406 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDmkpxkktUBChAAr1Gx7fo9iUt7ArzUQuoqLA==", 2023-06-28T20:08:13.406 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.406 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-06-28T20:08:13.406 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.406 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.407 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.407 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-rgw", 2023-06-28T20:08:13.407 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDmkpxk3t8BChAAEq9fb/TdyE79/j+7JhIMzg==", 2023-06-28T20:08:13.407 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.407 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-rgw" 2023-06-28T20:08:13.407 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.407 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.408 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.408 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2023-06-28T20:08:13.408 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDCkpxkJPYtNRAAoSegj6HmD6qDRmhhE0VNug==", 2023-06-28T20:08:13.408 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.408 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-06-28T20:08:13.408 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-06-28T20:08:13.408 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-06-28T20:08:13.408 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.409 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:13.409 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:13.409 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2023-06-28T20:08:13.409 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQDCkpxk+Gz8LxAAeucI0skt1Rn3ga6yMBwuzA==", 2023-06-28T20:08:13.409 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2023-06-28T20:08:13.409 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2023-06-28T20:08:13.409 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2023-06-28T20:08:13.410 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2023-06-28T20:08:13.410 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.410 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:13.410 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2023-06-28T20:08:13.410 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-06-28T20:08:13.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.376+0000 7f6621c13700 1 -- 172.21.15.43:0/992472304 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f65fc06ace0 msgr2=0x7f65fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.376+0000 7f6621c13700 1 --2- 172.21.15.43:0/992472304 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f65fc06ace0 0x7f65fc06d190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f661c0ae670 tx=0x7f6604008030 comp rx=0 tx=0).stop 2023-06-28T20:08:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.376+0000 7f6621c13700 1 -- 172.21.15.43:0/992472304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f661c06ce80 msgr2=0x7f661c15f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.376+0000 7f6621c13700 1 --2- 172.21.15.43:0/992472304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f661c06ce80 0x7f661c15f4c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f660c00ebe0 tx=0x7f660c009e20 comp rx=0 tx=0).stop 2023-06-28T20:08:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.376+0000 7f6621c13700 1 -- 172.21.15.43:0/992472304 shutdown_connections 2023-06-28T20:08:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.376+0000 7f6621c13700 1 --2- 172.21.15.43:0/992472304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f661c0cbc40 0x7f661c0c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.376+0000 7f6621c13700 1 --2- 172.21.15.43:0/992472304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f661c06ce80 0x7f661c15f4c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.376+0000 7f6621c13700 1 --2- 172.21.15.43:0/992472304 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f65fc06ace0 0x7f65fc06d190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.376+0000 7f6621c13700 1 --2- 172.21.15.43:0/992472304 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f661c0af060 0x7f661c159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.376+0000 7f6621c13700 1 -- 172.21.15.43:0/992472304 >> 172.21.15.43:0/992472304 conn(0x7f661c0b1720 msgr2=0x7f661c15afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.376+0000 7f6621c13700 1 -- 172.21.15.43:0/992472304 shutdown_connections 2023-06-28T20:08:13.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.376+0000 7f6621c13700 1 -- 172.21.15.43:0/992472304 wait complete. 2023-06-28T20:08:13.413 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-06-28T20:08:13.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.535+0000 7fba7dd5f700 1 Processor -- start 2023-06-28T20:08:13.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.535+0000 7fba7dd5f700 1 -- start start 2023-06-28T20:08:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.536+0000 7fba7dd5f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fba780b1f90 0x7fba78070900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.536+0000 7fba7dd5f700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fba78070e40 0x7fba7815df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.536+0000 7fba7dd5f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba78158290 con 0x7fba78157ec0 2023-06-28T20:08:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.536+0000 7fba7dd5f700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba7815a920 con 0x7fba780b1f90 2023-06-28T20:08:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.536+0000 7fba7dd5f700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba7815ab30 con 0x7fba78070e40 2023-06-28T20:08:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.536+0000 7fba76ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fba780b1f90 0x7fba78070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.536+0000 7fba777fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fba78157ec0 0x7fba780b1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43918/0 (socket says 172.21.15.43:43918) 2023-06-28T20:08:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.536+0000 7fba777fe700 1 -- 172.21.15.43:0/1046525712 learned_addr learned my addr 172.21.15.43:0/1046525712 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.536+0000 7fba767fc700 1 -- 172.21.15.43:0/1046525712 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 73142995 0 0) 0x7fba78158290 con 0x7fba78157ec0 2023-06-28T20:08:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.536+0000 7fba77fff700 1 --2- 172.21.15.43:0/1046525712 >> v2:172.21.15.178:3300/0 conn(0x7fba78070e40 0x7fba7815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.536+0000 7fba767fc700 1 -- 172.21.15.43:0/1046525712 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba640035b0 con 0x7fba78157ec0 2023-06-28T20:08:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.536+0000 7fba76ffd700 1 -- 172.21.15.43:0/1046525712 >> v2:172.21.15.178:3300/0 conn(0x7fba78070e40 msgr2=0x7fba7815df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.536+0000 7fba76ffd700 1 --2- 172.21.15.43:0/1046525712 >> v2:172.21.15.178:3300/0 conn(0x7fba78070e40 0x7fba7815df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.536+0000 7fba76ffd700 1 -- 172.21.15.43:0/1046525712 >> v1:172.21.15.43:6789/0 conn(0x7fba78157ec0 legacy=0x7fba780b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:13.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.537+0000 7fba76ffd700 1 -- 172.21.15.43:0/1046525712 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7815ae10 con 0x7fba780b1f90 2023-06-28T20:08:13.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.537+0000 7fba76ffd700 1 --2- 172.21.15.43:0/1046525712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba780b1f90 0x7fba78070900 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fba78070660 tx=0x7fba60009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de60674f1cbf0013 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:13.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.537+0000 7fba767fc700 1 -- 172.21.15.43:0/1046525712 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba60011830 con 0x7fba780b1f90 2023-06-28T20:08:13.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.537+0000 7fba767fc700 1 -- 172.21.15.43:0/1046525712 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba60011990 con 0x7fba780b1f90 2023-06-28T20:08:13.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.537+0000 7fba767fc700 1 -- 172.21.15.43:0/1046525712 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba60011d20 con 0x7fba780b1f90 2023-06-28T20:08:13.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.537+0000 7fba7dd5f700 1 -- 172.21.15.43:0/1046525712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba780b1f90 msgr2=0x7fba78070900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:13.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.537+0000 7fba7dd5f700 1 --2- 172.21.15.43:0/1046525712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba780b1f90 0x7fba78070900 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fba78070660 tx=0x7fba60009b20 comp rx=0 tx=0).stop 2023-06-28T20:08:13.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.537+0000 7fba7dd5f700 1 -- 172.21.15.43:0/1046525712 shutdown_connections 2023-06-28T20:08:13.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.537+0000 7fba7dd5f700 1 --2- 172.21.15.43:0/1046525712 >> v2:172.21.15.178:3300/0 conn(0x7fba78070e40 0x7fba7815df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.537+0000 7fba7dd5f700 1 --2- 172.21.15.43:0/1046525712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba780b1f90 0x7fba78070900 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.537+0000 7fba7dd5f700 1 -- 172.21.15.43:0/1046525712 >> 172.21.15.43:0/1046525712 conn(0x7fba7806b000 msgr2=0x7fba7806b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:13.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.537+0000 7fba7dd5f700 1 -- 172.21.15.43:0/1046525712 shutdown_connections 2023-06-28T20:08:13.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.537+0000 7fba7dd5f700 1 -- 172.21.15.43:0/1046525712 wait complete. 2023-06-28T20:08:13.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.539+0000 7fba7dd5f700 1 Processor -- start 2023-06-28T20:08:13.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.539+0000 7fba7dd5f700 1 -- start start 2023-06-28T20:08:13.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.540+0000 7fba7dd5f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba78070e40 0x7fba781ef4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:13.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.540+0000 7fba7dd5f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba780b1f90 0x7fba780afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:13.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.540+0000 7fba7dd5f700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba78157ec0 0x7fba78156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:13.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.540+0000 7fba7dd5f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba780caf00 con 0x7fba780b1f90 2023-06-28T20:08:13.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.540+0000 7fba7dd5f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba780cb1a0 con 0x7fba78070e40 2023-06-28T20:08:13.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.540+0000 7fba7dd5f700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba780cb480 con 0x7fba78157ec0 2023-06-28T20:08:13.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.540+0000 7fba76ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba780b1f90 0x7fba780afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:13.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.540+0000 7fba76ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba780b1f90 0x7fba780afa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54740/0 (socket says 172.21.15.43:54740) 2023-06-28T20:08:13.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.540+0000 7fba76ffd700 1 -- 172.21.15.43:0/1012613436 learned_addr learned my addr 172.21.15.43:0/1012613436 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:13.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.540+0000 7fba777fe700 1 --2- 172.21.15.43:0/1012613436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba78070e40 0x7fba781ef4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:13.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.540+0000 7fba77fff700 1 --2- 172.21.15.43:0/1012613436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba78157ec0 0x7fba78156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:13.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.540+0000 7fba76ffd700 1 -- 172.21.15.43:0/1012613436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba78157ec0 msgr2=0x7fba78156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:13.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.540+0000 7fba76ffd700 1 --2- 172.21.15.43:0/1012613436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba78157ec0 0x7fba78156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.540+0000 7fba76ffd700 1 -- 172.21.15.43:0/1012613436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba78070e40 msgr2=0x7fba781ef4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:13.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.540+0000 7fba76ffd700 1 --2- 172.21.15.43:0/1012613436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba78070e40 0x7fba781ef4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.540+0000 7fba76ffd700 1 -- 172.21.15.43:0/1012613436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba600094a0 con 0x7fba780b1f90 2023-06-28T20:08:13.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.540+0000 7fba777fe700 1 --2- 172.21.15.43:0/1012613436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba78070e40 0x7fba781ef4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:13.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.540+0000 7fba76ffd700 1 --2- 172.21.15.43:0/1012613436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba780b1f90 0x7fba780afa70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fba60009b60 tx=0x7fba60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:13.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.540+0000 7fba6ffff700 1 -- 172.21.15.43:0/1012613436 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba600036a0 con 0x7fba780b1f90 2023-06-28T20:08:13.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.540+0000 7fba6ffff700 1 -- 172.21.15.43:0/1012613436 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6001f070 con 0x7fba780b1f90 2023-06-28T20:08:13.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.540+0000 7fba6ffff700 1 -- 172.21.15.43:0/1012613436 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba60003a40 con 0x7fba780b1f90 2023-06-28T20:08:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.540+0000 7fba7dd5f700 1 -- 172.21.15.43:0/1012613436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba780cb760 con 0x7fba780b1f90 2023-06-28T20:08:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.540+0000 7fba7dd5f700 1 -- 172.21.15.43:0/1012613436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba781ef9f0 con 0x7fba780b1f90 2023-06-28T20:08:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.541+0000 7fba7dd5f700 1 -- 172.21.15.43:0/1012613436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba780728b0 con 0x7fba780b1f90 2023-06-28T20:08:13.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.542+0000 7fba6ffff700 1 -- 172.21.15.43:0/1012613436 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fba60003ba0 con 0x7fba780b1f90 2023-06-28T20:08:13.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.542+0000 7fba6ffff700 1 --2- 172.21.15.43:0/1012613436 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fba5806ace0 0x7fba5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:13.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.542+0000 7fba6ffff700 1 -- 172.21.15.43:0/1012613436 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fba60021070 con 0x7fba780b1f90 2023-06-28T20:08:13.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.542+0000 7fba777fe700 1 --2- 172.21.15.43:0/1012613436 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fba5806ace0 0x7fba5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:13.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.543+0000 7fba777fe700 1 --2- 172.21.15.43:0/1012613436 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fba5806ace0 0x7fba5806d190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fba680011e0 tx=0x7fba68009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:13.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.547+0000 7fba6ffff700 1 -- 172.21.15.43:0/1012613436 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fba60055e00 con 0x7fba780b1f90 2023-06-28T20:08:13.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.830+0000 7fba7dd5f700 1 -- 172.21.15.43:0/1012613436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fba780b5a70 con 0x7fba780b1f90 2023-06-28T20:08:13.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.833+0000 7fba6ffff700 1 -- 172.21.15.43:0/1012613436 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fba60059420 con 0x7fba780b1f90 2023-06-28T20:08:13.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.835+0000 7fba7dd5f700 1 -- 172.21.15.43:0/1012613436 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fba5806ace0 msgr2=0x7fba5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:13.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.835+0000 7fba7dd5f700 1 --2- 172.21.15.43:0/1012613436 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fba5806ace0 0x7fba5806d190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fba680011e0 tx=0x7fba68009040 comp rx=0 tx=0).stop 2023-06-28T20:08:13.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.835+0000 7fba7dd5f700 1 -- 172.21.15.43:0/1012613436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba780b1f90 msgr2=0x7fba780afa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:13.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.835+0000 7fba7dd5f700 1 --2- 172.21.15.43:0/1012613436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba780b1f90 0x7fba780afa70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fba60009b60 tx=0x7fba60004750 comp rx=0 tx=0).stop 2023-06-28T20:08:13.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.836+0000 7fba7dd5f700 1 -- 172.21.15.43:0/1012613436 shutdown_connections 2023-06-28T20:08:13.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.836+0000 7fba7dd5f700 1 --2- 172.21.15.43:0/1012613436 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fba5806ace0 0x7fba5806d190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.836+0000 7fba7dd5f700 1 --2- 172.21.15.43:0/1012613436 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fba78157ec0 0x7fba78156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.836+0000 7fba7dd5f700 1 --2- 172.21.15.43:0/1012613436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba78070e40 0x7fba781ef4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.836+0000 7fba7dd5f700 1 --2- 172.21.15.43:0/1012613436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba780b1f90 0x7fba780afa70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.836+0000 7fba7dd5f700 1 -- 172.21.15.43:0/1012613436 >> 172.21.15.43:0/1012613436 conn(0x7fba7806b000 msgr2=0x7fba780ae6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:13.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.836+0000 7fba7dd5f700 1 -- 172.21.15.43:0/1012613436 shutdown_connections 2023-06-28T20:08:13.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.836+0000 7fba7dd5f700 1 -- 172.21.15.43:0/1012613436 wait complete. 2023-06-28T20:08:13.852 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-06-28T20:08:13.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.974+0000 7f2b9d4cb700 1 Processor -- start 2023-06-28T20:08:13.979 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.974+0000 7f2b9d4cb700 1 -- start start 2023-06-28T20:08:13.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.975+0000 7f2b9d4cb700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2b980aec90 0x7f2b98160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:13.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.975+0000 7f2b9d4cb700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2b980af060 0x7f2b98070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:13.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.975+0000 7f2b9d4cb700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b98160250 con 0x7f2b9806ccc0 2023-06-28T20:08:13.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.975+0000 7f2b9d4cb700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b9815cb30 con 0x7f2b980aec90 2023-06-28T20:08:13.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.975+0000 7f2b9d4cb700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b98071410 con 0x7f2b980af060 2023-06-28T20:08:13.980 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.975+0000 7f2b96ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2b980aec90 0x7f2b98160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:13.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.975+0000 7f2b96ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2b980aec90 0x7f2b98160cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54746/0 (socket says 172.21.15.43:54746) 2023-06-28T20:08:13.981 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.975+0000 7f2b96ffd700 1 -- 172.21.15.43:0/1620285854 learned_addr learned my addr 172.21.15.43:0/1620285854 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:13.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.976+0000 7f2b96ffd700 1 -- 172.21.15.43:0/1620285854 >> v2:172.21.15.178:3300/0 conn(0x7f2b980af060 msgr2=0x7f2b98070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:13.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.976+0000 7f2b967fc700 1 --2- 172.21.15.43:0/1620285854 >> v2:172.21.15.178:3300/0 conn(0x7f2b980af060 0x7f2b98070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:13.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.976+0000 7f2b96ffd700 1 --2- 172.21.15.43:0/1620285854 >> v2:172.21.15.178:3300/0 conn(0x7f2b980af060 0x7f2b98070bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.976+0000 7f2b96ffd700 1 -- 172.21.15.43:0/1620285854 >> v1:172.21.15.178:6789/0 conn(0x7f2b9806ccc0 legacy=0x7f2b980b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:13.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.976+0000 7f2b96ffd700 1 -- 172.21.15.43:0/1620285854 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b981604f0 con 0x7f2b980aec90 2023-06-28T20:08:13.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.976+0000 7f2b96ffd700 1 --2- 172.21.15.43:0/1620285854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b980aec90 0x7f2b98160cf0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2b980d0310 tx=0x7f2b8c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f39749a6d4f5aa server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:13.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.976+0000 7f2b95ffb700 1 -- 172.21.15.43:0/1620285854 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c004230 con 0x7f2b980aec90 2023-06-28T20:08:13.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.976+0000 7f2b95ffb700 1 -- 172.21.15.43:0/1620285854 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8c004390 con 0x7f2b980aec90 2023-06-28T20:08:13.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.976+0000 7f2b95ffb700 1 -- 172.21.15.43:0/1620285854 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c0036a0 con 0x7f2b980aec90 2023-06-28T20:08:13.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.976+0000 7f2b9d4cb700 1 -- 172.21.15.43:0/1620285854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b980aec90 msgr2=0x7f2b98160cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:13.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.976+0000 7f2b9d4cb700 1 --2- 172.21.15.43:0/1620285854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b980aec90 0x7f2b98160cf0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2b980d0310 tx=0x7f2b8c009bd0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.976+0000 7f2b9d4cb700 1 -- 172.21.15.43:0/1620285854 shutdown_connections 2023-06-28T20:08:13.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.976+0000 7f2b9d4cb700 1 --2- 172.21.15.43:0/1620285854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b980aec90 0x7f2b98160cf0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.976+0000 7f2b9d4cb700 1 --2- 172.21.15.43:0/1620285854 >> v2:172.21.15.178:3300/0 conn(0x7f2b980af060 0x7f2b98070bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.976+0000 7f2b9d4cb700 1 -- 172.21.15.43:0/1620285854 >> 172.21.15.43:0/1620285854 conn(0x7f2b9806b000 msgr2=0x7f2b9806b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:13.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.977+0000 7f2b9d4cb700 1 -- 172.21.15.43:0/1620285854 shutdown_connections 2023-06-28T20:08:13.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.977+0000 7f2b9d4cb700 1 -- 172.21.15.43:0/1620285854 wait complete. 2023-06-28T20:08:13.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.978+0000 7f2b9d4cb700 1 Processor -- start 2023-06-28T20:08:13.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.978+0000 7f2b9d4cb700 1 -- start start 2023-06-28T20:08:13.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.979+0000 7f2b9d4cb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b9806ccc0 0x7f2b980b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:13.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.979+0000 7f2b9d4cb700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b980aec90 0x7f2b981f2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:13.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.979+0000 7f2b9d4cb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b980af060 0x7f2b981f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:13.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.979+0000 7f2b9d4cb700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b980b5190 con 0x7f2b980af060 2023-06-28T20:08:13.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.979+0000 7f2b9d4cb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b980b4330 con 0x7f2b9806ccc0 2023-06-28T20:08:13.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.979+0000 7f2b9d4cb700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b981f2c60 con 0x7f2b980aec90 2023-06-28T20:08:13.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.979+0000 7f2b977fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b980af060 0x7f2b981f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:13.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.979+0000 7f2b977fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b980af060 0x7f2b981f4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54756/0 (socket says 172.21.15.43:54756) 2023-06-28T20:08:13.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.979+0000 7f2b977fe700 1 -- 172.21.15.43:0/3109698111 learned_addr learned my addr 172.21.15.43:0/3109698111 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:13.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.979+0000 7f2b967fc700 1 --2- 172.21.15.43:0/3109698111 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b980aec90 0x7f2b981f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:13.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.979+0000 7f2b96ffd700 1 --2- 172.21.15.43:0/3109698111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b9806ccc0 0x7f2b980b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:13.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.979+0000 7f2b977fe700 1 -- 172.21.15.43:0/3109698111 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b980aec90 msgr2=0x7f2b981f2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:13.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.979+0000 7f2b977fe700 1 --2- 172.21.15.43:0/3109698111 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b980aec90 0x7f2b981f2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.979+0000 7f2b977fe700 1 -- 172.21.15.43:0/3109698111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b9806ccc0 msgr2=0x7f2b980b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:13.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.979+0000 7f2b977fe700 1 --2- 172.21.15.43:0/3109698111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b9806ccc0 0x7f2b980b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:13.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.979+0000 7f2b977fe700 1 -- 172.21.15.43:0/3109698111 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c009550 con 0x7f2b980af060 2023-06-28T20:08:13.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.979+0000 7f2b967fc700 1 --2- 172.21.15.43:0/3109698111 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b980aec90 0x7f2b981f2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:13.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.979+0000 7f2b96ffd700 1 --2- 172.21.15.43:0/3109698111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b9806ccc0 0x7f2b980b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:13.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.979+0000 7f2b977fe700 1 --2- 172.21.15.43:0/3109698111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b980af060 0x7f2b981f4a80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000c7d0 tx=0x7f2b9000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:13.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.979+0000 7f2b7f7fe700 1 -- 172.21.15.43:0/3109698111 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b90003e10 con 0x7f2b980af060 2023-06-28T20:08:13.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.979+0000 7f2b7f7fe700 1 -- 172.21.15.43:0/3109698111 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b90005070 con 0x7f2b980af060 2023-06-28T20:08:13.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.979+0000 7f2b9d4cb700 1 -- 172.21.15.43:0/3109698111 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b980b4580 con 0x7f2b980af060 2023-06-28T20:08:13.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.979+0000 7f2b7f7fe700 1 -- 172.21.15.43:0/3109698111 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b90005420 con 0x7f2b980af060 2023-06-28T20:08:13.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.980+0000 7f2b9d4cb700 1 -- 172.21.15.43:0/3109698111 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b980b4b40 con 0x7f2b980af060 2023-06-28T20:08:13.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.980+0000 7f2b9d4cb700 1 -- 172.21.15.43:0/3109698111 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b980728b0 con 0x7f2b980af060 2023-06-28T20:08:13.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.981+0000 7f2b7f7fe700 1 -- 172.21.15.43:0/3109698111 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2b90005580 con 0x7f2b980af060 2023-06-28T20:08:13.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.981+0000 7f2b7f7fe700 1 --2- 172.21.15.43:0/3109698111 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2b8406ace0 0x7f2b8406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:13.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.982+0000 7f2b7f7fe700 1 -- 172.21.15.43:0/3109698111 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2b900888a0 con 0x7f2b980af060 2023-06-28T20:08:13.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.982+0000 7f2b96ffd700 1 --2- 172.21.15.43:0/3109698111 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2b8406ace0 0x7f2b8406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:13.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.982+0000 7f2b96ffd700 1 --2- 172.21.15.43:0/3109698111 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2b8406ace0 0x7f2b8406d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c009130 tx=0x7f2b8c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:13.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:13.986+0000 7f2b7f7fe700 1 -- 172.21.15.43:0/3109698111 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2b90053ca0 con 0x7f2b980af060 2023-06-28T20:08:14.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.265+0000 7f2b9d4cb700 1 -- 172.21.15.43:0/3109698111 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f2b9806d9c0 con 0x7f2b980af060 2023-06-28T20:08:14.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.268+0000 7f2b7f7fe700 1 -- 172.21.15.43:0/3109698111 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2b90017020 con 0x7f2b980af060 2023-06-28T20:08:14.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.270+0000 7f2b9d4cb700 1 -- 172.21.15.43:0/3109698111 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2b8406ace0 msgr2=0x7f2b8406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:14.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.270+0000 7f2b9d4cb700 1 --2- 172.21.15.43:0/3109698111 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2b8406ace0 0x7f2b8406d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c009130 tx=0x7f2b8c008030 comp rx=0 tx=0).stop 2023-06-28T20:08:14.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.270+0000 7f2b9d4cb700 1 -- 172.21.15.43:0/3109698111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b980af060 msgr2=0x7f2b981f4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:14.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.270+0000 7f2b9d4cb700 1 --2- 172.21.15.43:0/3109698111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b980af060 0x7f2b981f4a80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000c7d0 tx=0x7f2b9000cb90 comp rx=0 tx=0).stop 2023-06-28T20:08:14.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.270+0000 7f2b9d4cb700 1 -- 172.21.15.43:0/3109698111 shutdown_connections 2023-06-28T20:08:14.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.270+0000 7f2b9d4cb700 1 --2- 172.21.15.43:0/3109698111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b980af060 0x7f2b981f4a80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:14.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.270+0000 7f2b9d4cb700 1 --2- 172.21.15.43:0/3109698111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b9806ccc0 0x7f2b980b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:14.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.270+0000 7f2b9d4cb700 1 --2- 172.21.15.43:0/3109698111 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2b8406ace0 0x7f2b8406d190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:14.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.270+0000 7f2b9d4cb700 1 --2- 172.21.15.43:0/3109698111 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2b980aec90 0x7f2b981f2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:14.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.270+0000 7f2b9d4cb700 1 -- 172.21.15.43:0/3109698111 >> 172.21.15.43:0/3109698111 conn(0x7f2b9806b000 msgr2=0x7f2b980b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:14.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.270+0000 7f2b9d4cb700 1 -- 172.21.15.43:0/3109698111 shutdown_connections 2023-06-28T20:08:14.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.270+0000 7f2b9d4cb700 1 -- 172.21.15.43:0/3109698111 wait complete. 2023-06-28T20:08:14.286 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-06-28T20:08:14.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.409+0000 7f19d1ee9700 1 Processor -- start 2023-06-28T20:08:14.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.409+0000 7f19d1ee9700 1 -- start start 2023-06-28T20:08:14.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.409+0000 7f19d1ee9700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19cc06a830 0x7f19cc0c9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:14.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.409+0000 7f19d1ee9700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f19cc06ac00 0x7f19cc06b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:14.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.409+0000 7f19d1ee9700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19cc0ce370 con 0x7f19cc069ef0 2023-06-28T20:08:14.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.409+0000 7f19d1ee9700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19cc06b070 con 0x7f19cc06ac00 2023-06-28T20:08:14.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.409+0000 7f19d1ee9700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc0ce610 con 0x7f19cc06a830 2023-06-28T20:08:14.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.409+0000 7f19cbfff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f19cc069ef0 0x7f19cc06c0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43950/0 (socket says 172.21.15.43:43950) 2023-06-28T20:08:14.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.410+0000 7f19cbfff700 1 -- 172.21.15.43:0/2117742198 learned_addr learned my addr 172.21.15.43:0/2117742198 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:14.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.410+0000 7f19cb7fe700 1 --2- 172.21.15.43:0/2117742198 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19cc06a830 0x7f19cc0c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:14.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.410+0000 7f19caffd700 1 --2- 172.21.15.43:0/2117742198 >> v2:172.21.15.178:3300/0 conn(0x7f19cc06ac00 0x7f19cc06b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:14.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.410+0000 7f19ca7fc700 1 -- 172.21.15.43:0/2117742198 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1928769707 0 0) 0x7f19cc0ce370 con 0x7f19cc069ef0 2023-06-28T20:08:14.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.410+0000 7f19ca7fc700 1 -- 172.21.15.43:0/2117742198 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19b40035b0 con 0x7f19cc069ef0 2023-06-28T20:08:14.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.410+0000 7f19cb7fe700 1 -- 172.21.15.43:0/2117742198 >> v2:172.21.15.178:3300/0 conn(0x7f19cc06ac00 msgr2=0x7f19cc06b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:14.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.410+0000 7f19cb7fe700 1 --2- 172.21.15.43:0/2117742198 >> v2:172.21.15.178:3300/0 conn(0x7f19cc06ac00 0x7f19cc06b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:14.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.410+0000 7f19ca7fc700 1 -- 172.21.15.43:0/2117742198 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4183748915 0 0) 0x7f19b40035b0 con 0x7f19cc069ef0 2023-06-28T20:08:14.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.410+0000 7f19cb7fe700 1 -- 172.21.15.43:0/2117742198 >> v1:172.21.15.43:6789/0 conn(0x7f19cc069ef0 legacy=0x7f19cc06c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:14.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.410+0000 7f19cb7fe700 1 -- 172.21.15.43:0/2117742198 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19cc0ce8f0 con 0x7f19cc06a830 2023-06-28T20:08:14.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.410+0000 7f19caffd700 1 --2- 172.21.15.43:0/2117742198 >> v2:172.21.15.178:3300/0 conn(0x7f19cc06ac00 0x7f19cc06b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:08:14.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.410+0000 7f19cb7fe700 1 --2- 172.21.15.43:0/2117742198 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19cc06a830 0x7f19cc0c9900 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f19bc00bae0 tx=0x7f19bc00bdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2962bcc07d21bee5 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:14.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.410+0000 7f19ca7fc700 1 -- 172.21.15.43:0/2117742198 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c40019f0 con 0x7f19cc06a830 2023-06-28T20:08:14.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.410+0000 7f19ca7fc700 1 -- 172.21.15.43:0/2117742198 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19bc00da70 con 0x7f19cc06a830 2023-06-28T20:08:14.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.411+0000 7f19ca7fc700 1 -- 172.21.15.43:0/2117742198 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc00de20 con 0x7f19cc06a830 2023-06-28T20:08:14.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.411+0000 7f19d1ee9700 1 -- 172.21.15.43:0/2117742198 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19cc06a830 msgr2=0x7f19cc0c9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:14.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.411+0000 7f19d1ee9700 1 --2- 172.21.15.43:0/2117742198 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19cc06a830 0x7f19cc0c9900 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f19bc00bae0 tx=0x7f19bc00bdf0 comp rx=0 tx=0).stop 2023-06-28T20:08:14.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.411+0000 7f19d1ee9700 1 -- 172.21.15.43:0/2117742198 shutdown_connections 2023-06-28T20:08:14.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.411+0000 7f19d1ee9700 1 --2- 172.21.15.43:0/2117742198 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19cc06a830 0x7f19cc0c9900 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:14.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.411+0000 7f19d1ee9700 1 --2- 172.21.15.43:0/2117742198 >> v2:172.21.15.178:3300/0 conn(0x7f19cc06ac00 0x7f19cc06b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:14.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.411+0000 7f19d1ee9700 1 -- 172.21.15.43:0/2117742198 >> 172.21.15.43:0/2117742198 conn(0x7f19cc0b1a50 msgr2=0x7f19cc0cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:14.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.411+0000 7f19d1ee9700 1 -- 172.21.15.43:0/2117742198 shutdown_connections 2023-06-28T20:08:14.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.411+0000 7f19d1ee9700 1 -- 172.21.15.43:0/2117742198 wait complete. 2023-06-28T20:08:14.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.412+0000 7f19d1ee9700 1 Processor -- start 2023-06-28T20:08:14.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.412+0000 7f19d1ee9700 1 -- start start 2023-06-28T20:08:14.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.413+0000 7f19d1ee9700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19cc069ef0 0x7f19cc147a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:14.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.413+0000 7f19d1ee9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19cc06a830 0x7f19cc1f0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:14.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.413+0000 7f19d1ee9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19cc06ac00 0x7f19cc0c2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:14.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.413+0000 7f19d1ee9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc147610 con 0x7f19cc06a830 2023-06-28T20:08:14.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.413+0000 7f19d1ee9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc1f1050 con 0x7f19cc06ac00 2023-06-28T20:08:14.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.413+0000 7f19d1ee9700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc0c37d0 con 0x7f19cc069ef0 2023-06-28T20:08:14.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.413+0000 7f19caffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19cc06a830 0x7f19cc1f0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:14.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.413+0000 7f19caffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19cc06a830 0x7f19cc1f0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54768/0 (socket says 172.21.15.43:54768) 2023-06-28T20:08:14.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.413+0000 7f19caffd700 1 -- 172.21.15.43:0/670392682 learned_addr learned my addr 172.21.15.43:0/670392682 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:14.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.413+0000 7f19cbfff700 1 --2- 172.21.15.43:0/670392682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19cc06ac00 0x7f19cc0c2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:14.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.413+0000 7f19cb7fe700 1 --2- 172.21.15.43:0/670392682 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19cc069ef0 0x7f19cc147a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:14.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.414+0000 7f19caffd700 1 -- 172.21.15.43:0/670392682 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19cc069ef0 msgr2=0x7f19cc147a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:14.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.414+0000 7f19caffd700 1 --2- 172.21.15.43:0/670392682 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19cc069ef0 0x7f19cc147a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:14.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.414+0000 7f19caffd700 1 -- 172.21.15.43:0/670392682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19cc06ac00 msgr2=0x7f19cc0c2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:14.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.414+0000 7f19caffd700 1 --2- 172.21.15.43:0/670392682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19cc06ac00 0x7f19cc0c2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:14.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.414+0000 7f19caffd700 1 -- 172.21.15.43:0/670392682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c00094a0 con 0x7f19cc06a830 2023-06-28T20:08:14.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.414+0000 7f19cbfff700 1 --2- 172.21.15.43:0/670392682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19cc06ac00 0x7f19cc0c2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:14.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.414+0000 7f19caffd700 1 --2- 172.21.15.43:0/670392682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19cc06a830 0x7f19cc1f0b10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f19c000ebe0 tx=0x7f19c0009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:14.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.414+0000 7f19b3fff700 1 -- 172.21.15.43:0/670392682 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c000c910 con 0x7f19cc06a830 2023-06-28T20:08:14.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.414+0000 7f19d1ee9700 1 -- 172.21.15.43:0/670392682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19bc00b740 con 0x7f19cc06a830 2023-06-28T20:08:14.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.414+0000 7f19b3fff700 1 -- 172.21.15.43:0/670392682 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c000ca70 con 0x7f19cc06a830 2023-06-28T20:08:14.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.414+0000 7f19b3fff700 1 -- 172.21.15.43:0/670392682 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c000cde0 con 0x7f19cc06a830 2023-06-28T20:08:14.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.414+0000 7f19d1ee9700 1 -- 172.21.15.43:0/670392682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19cc147230 con 0x7f19cc06a830 2023-06-28T20:08:14.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.415+0000 7f19d1ee9700 1 -- 172.21.15.43:0/670392682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19cc147750 con 0x7f19cc06a830 2023-06-28T20:08:14.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.415+0000 7f19b3fff700 1 -- 172.21.15.43:0/670392682 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f19c0013470 con 0x7f19cc06a830 2023-06-28T20:08:14.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.416+0000 7f19b3fff700 1 --2- 172.21.15.43:0/670392682 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f19b806ace0 0x7f19b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:14.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.416+0000 7f19b3fff700 1 -- 172.21.15.43:0/670392682 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f19c0089330 con 0x7f19cc06a830 2023-06-28T20:08:14.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.416+0000 7f19cb7fe700 1 --2- 172.21.15.43:0/670392682 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f19b806ace0 0x7f19b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:14.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.416+0000 7f19cb7fe700 1 --2- 172.21.15.43:0/670392682 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f19b806ace0 0x7f19b806d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f19bc000c00 tx=0x7f19bc013000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:14.426 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.420+0000 7f19b3fff700 1 -- 172.21.15.43:0/670392682 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f19cc147750 con 0x7f19cc06a830 2023-06-28T20:08:14.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.700+0000 7f19d1ee9700 1 -- 172.21.15.43:0/670392682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f19cc147750 con 0x7f19cc06a830 2023-06-28T20:08:14.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.703+0000 7f19b3fff700 1 -- 172.21.15.43:0/670392682 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f19cc147750 con 0x7f19cc06a830 2023-06-28T20:08:14.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.705+0000 7f19d1ee9700 1 -- 172.21.15.43:0/670392682 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f19b806ace0 msgr2=0x7f19b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:14.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.705+0000 7f19d1ee9700 1 --2- 172.21.15.43:0/670392682 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f19b806ace0 0x7f19b806d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f19bc000c00 tx=0x7f19bc013000 comp rx=0 tx=0).stop 2023-06-28T20:08:14.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.705+0000 7f19d1ee9700 1 -- 172.21.15.43:0/670392682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19cc06a830 msgr2=0x7f19cc1f0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:14.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.705+0000 7f19d1ee9700 1 --2- 172.21.15.43:0/670392682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19cc06a830 0x7f19cc1f0b10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f19c000ebe0 tx=0x7f19c0009ec0 comp rx=0 tx=0).stop 2023-06-28T20:08:14.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.705+0000 7f19d1ee9700 1 -- 172.21.15.43:0/670392682 shutdown_connections 2023-06-28T20:08:14.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.705+0000 7f19d1ee9700 1 --2- 172.21.15.43:0/670392682 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19cc06ac00 0x7f19cc0c2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:14.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.705+0000 7f19d1ee9700 1 --2- 172.21.15.43:0/670392682 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f19b806ace0 0x7f19b806d190 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:14.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.705+0000 7f19d1ee9700 1 --2- 172.21.15.43:0/670392682 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f19cc069ef0 0x7f19cc147a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:14.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.705+0000 7f19d1ee9700 1 --2- 172.21.15.43:0/670392682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19cc06a830 0x7f19cc1f0b10 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:14.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.705+0000 7f19d1ee9700 1 -- 172.21.15.43:0/670392682 >> 172.21.15.43:0/670392682 conn(0x7f19cc0b1a50 msgr2=0x7f19cc0aec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:14.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.705+0000 7f19d1ee9700 1 -- 172.21.15.43:0/670392682 shutdown_connections 2023-06-28T20:08:14.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.705+0000 7f19d1ee9700 1 -- 172.21.15.43:0/670392682 wait complete. 2023-06-28T20:08:14.721 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-06-28T20:08:14.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.844+0000 7f77427b8700 1 Processor -- start 2023-06-28T20:08:14.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.845+0000 7f77427b8700 1 -- start start 2023-06-28T20:08:14.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.845+0000 7f77427b8700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f773c0af060 0x7f773c160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:14.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.845+0000 7f77427b8700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f773c160250 con 0x7f773c06ccc0 2023-06-28T20:08:14.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.845+0000 7f77427b8700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f773c071410 con 0x7f773c0aec90 2023-06-28T20:08:14.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.845+0000 7f77427b8700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f773c15cb30 con 0x7f773c0af060 2023-06-28T20:08:14.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.845+0000 7f773b7fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f773c0af060 0x7f773c160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:14.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.845+0000 7f7740d55700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f773c06ccc0 0x7f773c0b1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43964/0 (socket says 172.21.15.43:43964) 2023-06-28T20:08:14.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.845+0000 7f7740d55700 1 -- 172.21.15.43:0/2965967345 learned_addr learned my addr 172.21.15.43:0/2965967345 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:14.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.846+0000 7f773b7fe700 1 -- 172.21.15.43:0/2965967345 >> v1:172.21.15.178:6789/0 conn(0x7f773c0aec90 legacy=0x7f773c070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:14.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.846+0000 7f773b7fe700 1 -- 172.21.15.43:0/2965967345 >> v1:172.21.15.43:6789/0 conn(0x7f773c06ccc0 legacy=0x7f773c0b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:14.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.846+0000 7f773affd700 1 -- 172.21.15.43:0/2965967345 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1237409590 0 0) 0x7f773c160250 con 0x7f773c06ccc0 2023-06-28T20:08:14.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.846+0000 7f773b7fe700 1 -- 172.21.15.43:0/2965967345 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773c1604f0 con 0x7f773c0af060 2023-06-28T20:08:14.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.846+0000 7f773b7fe700 1 --2- 172.21.15.43:0/2965967345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773c0af060 0x7f773c160cf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7730009840 tx=0x7f7730009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de74f4a8a704a887 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:14.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.846+0000 7f773affd700 1 -- 172.21.15.43:0/2965967345 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7730011830 con 0x7f773c0af060 2023-06-28T20:08:14.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.846+0000 7f773affd700 1 -- 172.21.15.43:0/2965967345 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7730011990 con 0x7f773c0af060 2023-06-28T20:08:14.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.846+0000 7f773affd700 1 -- 172.21.15.43:0/2965967345 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7730011d20 con 0x7f773c0af060 2023-06-28T20:08:14.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.847+0000 7f77427b8700 1 -- 172.21.15.43:0/2965967345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773c0af060 msgr2=0x7f773c160cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:14.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.847+0000 7f77427b8700 1 --2- 172.21.15.43:0/2965967345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773c0af060 0x7f773c160cf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7730009840 tx=0x7f7730009b50 comp rx=0 tx=0).stop 2023-06-28T20:08:14.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.847+0000 7f77427b8700 1 -- 172.21.15.43:0/2965967345 shutdown_connections 2023-06-28T20:08:14.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.847+0000 7f77427b8700 1 --2- 172.21.15.43:0/2965967345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773c0af060 0x7f773c160cf0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:14.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.847+0000 7f77427b8700 1 -- 172.21.15.43:0/2965967345 >> 172.21.15.43:0/2965967345 conn(0x7f773c06b000 msgr2=0x7f773c06b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:14.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.847+0000 7f77427b8700 1 -- 172.21.15.43:0/2965967345 shutdown_connections 2023-06-28T20:08:14.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.847+0000 7f77427b8700 1 -- 172.21.15.43:0/2965967345 wait complete. 2023-06-28T20:08:14.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.848+0000 7f77427b8700 1 Processor -- start 2023-06-28T20:08:14.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.848+0000 7f77427b8700 1 -- start start 2023-06-28T20:08:14.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.849+0000 7f77427b8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f773c06ccc0 0x7f773c15e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:14.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.849+0000 7f77427b8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f773c0aec90 0x7f773c15eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:14.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.849+0000 7f77427b8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773c0af060 0x7f773c15f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:14.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.849+0000 7f77427b8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c1508b0 con 0x7f773c0af060 2023-06-28T20:08:14.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.849+0000 7f77427b8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773c15d050 con 0x7f773c0aec90 2023-06-28T20:08:14.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.849+0000 7f77427b8700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773c15d330 con 0x7f773c06ccc0 2023-06-28T20:08:14.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.849+0000 7f7740d55700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773c0af060 0x7f773c15f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:14.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.849+0000 7f7740d55700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773c0af060 0x7f773c15f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54784/0 (socket says 172.21.15.43:54784) 2023-06-28T20:08:14.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.849+0000 7f7740d55700 1 -- 172.21.15.43:0/2157345852 learned_addr learned my addr 172.21.15.43:0/2157345852 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:14.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.849+0000 7f773b7fe700 1 --2- 172.21.15.43:0/2157345852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f773c0aec90 0x7f773c15eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:14.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.849+0000 7f773bfff700 1 --2- 172.21.15.43:0/2157345852 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f773c06ccc0 0x7f773c15e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:14.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.849+0000 7f7740d55700 1 -- 172.21.15.43:0/2157345852 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f773c06ccc0 msgr2=0x7f773c15e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:14.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.849+0000 7f7740d55700 1 --2- 172.21.15.43:0/2157345852 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f773c06ccc0 0x7f773c15e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:14.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.849+0000 7f7740d55700 1 -- 172.21.15.43:0/2157345852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f773c0aec90 msgr2=0x7f773c15eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:14.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.849+0000 7f7740d55700 1 --2- 172.21.15.43:0/2157345852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f773c0aec90 0x7f773c15eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:14.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.849+0000 7f7740d55700 1 -- 172.21.15.43:0/2157345852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77300094a0 con 0x7f773c0af060 2023-06-28T20:08:14.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.849+0000 7f773bfff700 1 --2- 172.21.15.43:0/2157345852 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f773c06ccc0 0x7f773c15e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:14.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.850+0000 7f7740d55700 1 --2- 172.21.15.43:0/2157345852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773c0af060 0x7f773c15f530 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7734002230 tx=0x7f773400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:14.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.850+0000 7f7738ff9700 1 -- 172.21.15.43:0/2157345852 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773400ab70 con 0x7f773c0af060 2023-06-28T20:08:14.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.850+0000 7f7738ff9700 1 -- 172.21.15.43:0/2157345852 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773400acf0 con 0x7f773c0af060 2023-06-28T20:08:14.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.850+0000 7f7738ff9700 1 -- 172.21.15.43:0/2157345852 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7734007310 con 0x7f773c0af060 2023-06-28T20:08:14.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.850+0000 7f77427b8700 1 -- 172.21.15.43:0/2157345852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f773c15d610 con 0x7f773c0af060 2023-06-28T20:08:14.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.850+0000 7f77427b8700 1 -- 172.21.15.43:0/2157345852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773c15dbd0 con 0x7f773c0af060 2023-06-28T20:08:14.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.851+0000 7f77427b8700 1 -- 172.21.15.43:0/2157345852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f773c0728b0 con 0x7f773c0af060 2023-06-28T20:08:14.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.851+0000 7f7738ff9700 1 -- 172.21.15.43:0/2157345852 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7734007470 con 0x7f773c0af060 2023-06-28T20:08:14.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.851+0000 7f7738ff9700 1 --2- 172.21.15.43:0/2157345852 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f772806ace0 0x7f772806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:14.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.852+0000 7f7738ff9700 1 -- 172.21.15.43:0/2157345852 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7734088c00 con 0x7f773c0af060 2023-06-28T20:08:14.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.852+0000 7f773bfff700 1 --2- 172.21.15.43:0/2157345852 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f772806ace0 0x7f772806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:14.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.852+0000 7f773bfff700 1 --2- 172.21.15.43:0/2157345852 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f772806ace0 0x7f772806d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f772c000f60 tx=0x7f772c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:14.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:14.856+0000 7f7738ff9700 1 -- 172.21.15.43:0/2157345852 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f773408f080 con 0x7f773c0af060 2023-06-28T20:08:15.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.137+0000 7f77427b8700 1 -- 172.21.15.43:0/2157345852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f773c06d9c0 con 0x7f773c0af060 2023-06-28T20:08:15.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.141+0000 7f7738ff9700 1 -- 172.21.15.43:0/2157345852 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f77340180a0 con 0x7f773c0af060 2023-06-28T20:08:15.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.143+0000 7f77427b8700 1 -- 172.21.15.43:0/2157345852 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f772806ace0 msgr2=0x7f772806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:15.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.143+0000 7f77427b8700 1 --2- 172.21.15.43:0/2157345852 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f772806ace0 0x7f772806d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f772c000f60 tx=0x7f772c009040 comp rx=0 tx=0).stop 2023-06-28T20:08:15.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.143+0000 7f77427b8700 1 -- 172.21.15.43:0/2157345852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773c0af060 msgr2=0x7f773c15f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:15.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.143+0000 7f77427b8700 1 --2- 172.21.15.43:0/2157345852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773c0af060 0x7f773c15f530 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7734002230 tx=0x7f773400cf50 comp rx=0 tx=0).stop 2023-06-28T20:08:15.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.143+0000 7f77427b8700 1 -- 172.21.15.43:0/2157345852 shutdown_connections 2023-06-28T20:08:15.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.143+0000 7f77427b8700 1 --2- 172.21.15.43:0/2157345852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773c0af060 0x7f773c15f530 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:15.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.143+0000 7f77427b8700 1 --2- 172.21.15.43:0/2157345852 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f772806ace0 0x7f772806d190 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:15.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.143+0000 7f77427b8700 1 --2- 172.21.15.43:0/2157345852 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f773c06ccc0 0x7f773c15e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:15.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.143+0000 7f77427b8700 1 --2- 172.21.15.43:0/2157345852 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f773c0aec90 0x7f773c15eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:15.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.143+0000 7f77427b8700 1 -- 172.21.15.43:0/2157345852 >> 172.21.15.43:0/2157345852 conn(0x7f773c06b000 msgr2=0x7f773c0b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.143+0000 7f77427b8700 1 -- 172.21.15.43:0/2157345852 shutdown_connections 2023-06-28T20:08:15.149 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.143+0000 7f77427b8700 1 -- 172.21.15.43:0/2157345852 wait complete. 2023-06-28T20:08:15.159 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-06-28T20:08:15.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.281+0000 7f85f5cdc700 1 Processor -- start 2023-06-28T20:08:15.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.281+0000 7f85f5cdc700 1 -- start start 2023-06-28T20:08:15.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.282+0000 7f85f5cdc700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f85f00afdf0 0x7f85f006da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:15.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.282+0000 7f85f5cdc700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85f00ae950 con 0x7f85f00ceee0 2023-06-28T20:08:15.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.282+0000 7f85f5cdc700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85f015a9c0 con 0x7f85f015a5f0 2023-06-28T20:08:15.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.282+0000 7f85f5cdc700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f00add40 con 0x7f85f00afdf0 2023-06-28T20:08:15.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.282+0000 7f85eeffd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f85f00ceee0 0x7f85f006d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43976/0 (socket says 172.21.15.43:43976) 2023-06-28T20:08:15.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.282+0000 7f85eeffd700 1 -- 172.21.15.43:0/255240427 learned_addr learned my addr 172.21.15.43:0/255240427 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:15.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.282+0000 7f85effff700 1 --2- 172.21.15.43:0/255240427 >> v2:172.21.15.178:3300/0 conn(0x7f85f00afdf0 0x7f85f006da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:15.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.282+0000 7f85ee7fc700 1 -- 172.21.15.43:0/255240427 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2948981008 0 0) 0x7f85f00ae950 con 0x7f85f00ceee0 2023-06-28T20:08:15.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.282+0000 7f85ee7fc700 1 -- 172.21.15.43:0/255240427 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85dc0035b0 con 0x7f85f00ceee0 2023-06-28T20:08:15.290 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85ee7fc700 1 -- 172.21.15.43:0/255240427 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3759890385 0 0) 0x7f85dc0035b0 con 0x7f85f00ceee0 2023-06-28T20:08:15.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85ee7fc700 1 -- 172.21.15.43:0/255240427 >> v2:172.21.15.178:3300/0 conn(0x7f85f00afdf0 msgr2=0x7f85f006da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:15.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85ee7fc700 1 --2- 172.21.15.43:0/255240427 >> v2:172.21.15.178:3300/0 conn(0x7f85f00afdf0 0x7f85f006da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:15.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85ee7fc700 1 -- 172.21.15.43:0/255240427 >> v1:172.21.15.178:6789/0 conn(0x7f85f015a5f0 legacy=0x7f85f00ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:15.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85ee7fc700 1 -- 172.21.15.43:0/255240427 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f00ae020 con 0x7f85f00ceee0 2023-06-28T20:08:15.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85ee7fc700 1 -- 172.21.15.43:0/255240427 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f85d8001670 con 0x7f85f00ceee0 2023-06-28T20:08:15.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85ee7fc700 1 -- 172.21.15.43:0/255240427 >> v1:172.21.15.43:6789/0 conn(0x7f85f00ceee0 legacy=0x7f85f006d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:15.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85ee7fc700 1 --2- 172.21.15.43:0/255240427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85dc003a00 0x7f85dc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:15.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85ee7fc700 1 --2- 172.21.15.43:0/255240427 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85dc004850 0x7f85dc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:15.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85ee7fc700 1 --2- 172.21.15.43:0/255240427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85dc005200 0x7f85dc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:15.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85ee7fc700 1 -- 172.21.15.43:0/255240427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f00add40 con 0x7f85dc005200 2023-06-28T20:08:15.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85ee7fc700 1 -- 172.21.15.43:0/255240427 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc00c080 con 0x7f85dc003a00 2023-06-28T20:08:15.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85ee7fc700 1 -- 172.21.15.43:0/255240427 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc00c240 con 0x7f85dc004850 2023-06-28T20:08:15.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85effff700 1 --2- 172.21.15.43:0/255240427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85dc005200 0x7f85dc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:15.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85ef7fe700 1 --2- 172.21.15.43:0/255240427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85dc003a00 0x7f85dc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:15.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85eeffd700 1 --2- 172.21.15.43:0/255240427 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85dc004850 0x7f85dc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:15.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85effff700 1 -- 172.21.15.43:0/255240427 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85dc004850 msgr2=0x7f85dc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:15.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85effff700 1 --2- 172.21.15.43:0/255240427 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85dc004850 0x7f85dc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:15.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85effff700 1 -- 172.21.15.43:0/255240427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85dc003a00 msgr2=0x7f85dc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:15.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85effff700 1 --2- 172.21.15.43:0/255240427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85dc003a00 0x7f85dc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:15.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85effff700 1 -- 172.21.15.43:0/255240427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc0035b0 con 0x7f85dc005200 2023-06-28T20:08:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85eeffd700 1 --2- 172.21.15.43:0/255240427 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85dc004850 0x7f85dc004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85effff700 1 -- 172.21.15.43:0/255240427 reap_dead start 2023-06-28T20:08:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85effff700 1 --2- 172.21.15.43:0/255240427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85dc005200 0x7f85dc00ba10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009fc0 tx=0x7f85dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40140024c1dcc56 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85ee7fc700 1 -- 172.21.15.43:0/255240427 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e8003a20 con 0x7f85dc005200 2023-06-28T20:08:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85ee7fc700 1 -- 172.21.15.43:0/255240427 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e800f040 con 0x7f85dc005200 2023-06-28T20:08:15.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.283+0000 7f85ee7fc700 1 -- 172.21.15.43:0/255240427 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e8003de0 con 0x7f85dc005200 2023-06-28T20:08:15.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.284+0000 7f85f5cdc700 1 -- 172.21.15.43:0/255240427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85dc005200 msgr2=0x7f85dc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:15.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.284+0000 7f85f5cdc700 1 --2- 172.21.15.43:0/255240427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85dc005200 0x7f85dc00ba10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f85e8009fc0 tx=0x7f85dc005880 comp rx=0 tx=0).stop 2023-06-28T20:08:15.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.284+0000 7f85f5cdc700 1 -- 172.21.15.43:0/255240427 shutdown_connections 2023-06-28T20:08:15.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.284+0000 7f85f5cdc700 1 --2- 172.21.15.43:0/255240427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85dc005200 0x7f85dc00ba10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:15.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.284+0000 7f85f5cdc700 1 -- 172.21.15.43:0/255240427 >> 172.21.15.43:0/255240427 conn(0x7f85f00664b0 msgr2=0x7f85f00d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:15.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.284+0000 7f85f5cdc700 1 -- 172.21.15.43:0/255240427 shutdown_connections 2023-06-28T20:08:15.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.284+0000 7f85f5cdc700 1 -- 172.21.15.43:0/255240427 wait complete. 2023-06-28T20:08:15.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.286+0000 7f85f5cdc700 1 Processor -- start 2023-06-28T20:08:15.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.286+0000 7f85f5cdc700 1 -- start start 2023-06-28T20:08:15.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.287+0000 7f85f5cdc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85dc005200 0x7f85f00c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:15.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.287+0000 7f85f5cdc700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85f00b3240 0x7f85f00c4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:15.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.287+0000 7f85f5cdc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85f01e3c90 0x7f85f01e4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:15.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.287+0000 7f85f5cdc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f01efb90 con 0x7f85dc005200 2023-06-28T20:08:15.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.287+0000 7f85f5cdc700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f01e4640 con 0x7f85f01e3c90 2023-06-28T20:08:15.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.287+0000 7f85f5cdc700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f00c5250 con 0x7f85f00b3240 2023-06-28T20:08:15.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.287+0000 7f85ef7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85dc005200 0x7f85f00c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:15.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.287+0000 7f85ef7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85dc005200 0x7f85f00c70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54800/0 (socket says 172.21.15.43:54800) 2023-06-28T20:08:15.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.287+0000 7f85ef7fe700 1 -- 172.21.15.43:0/1078162106 learned_addr learned my addr 172.21.15.43:0/1078162106 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:15.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.287+0000 7f85effff700 1 --2- 172.21.15.43:0/1078162106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85f01e3c90 0x7f85f01e4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:15.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.287+0000 7f85eeffd700 1 --2- 172.21.15.43:0/1078162106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85f00b3240 0x7f85f00c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:15.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.287+0000 7f85ef7fe700 1 -- 172.21.15.43:0/1078162106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85f00b3240 msgr2=0x7f85f00c4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:15.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.287+0000 7f85ef7fe700 1 --2- 172.21.15.43:0/1078162106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85f00b3240 0x7f85f00c4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:15.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.287+0000 7f85ef7fe700 1 -- 172.21.15.43:0/1078162106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85f01e3c90 msgr2=0x7f85f01e4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:15.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.287+0000 7f85ef7fe700 1 --2- 172.21.15.43:0/1078162106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85f01e3c90 0x7f85f01e4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:15.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.287+0000 7f85ef7fe700 1 -- 172.21.15.43:0/1078162106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f00c5530 con 0x7f85dc005200 2023-06-28T20:08:15.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.287+0000 7f85effff700 1 --2- 172.21.15.43:0/1078162106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85f01e3c90 0x7f85f01e4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:15.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.287+0000 7f85ef7fe700 1 --2- 172.21.15.43:0/1078162106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85dc005200 0x7f85f00c70c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f85e000f820 tx=0x7f85e000fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:15.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.288+0000 7f85e7fff700 1 -- 172.21.15.43:0/1078162106 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e00052e0 con 0x7f85dc005200 2023-06-28T20:08:15.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.288+0000 7f85e7fff700 1 -- 172.21.15.43:0/1078162106 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e0005460 con 0x7f85dc005200 2023-06-28T20:08:15.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.288+0000 7f85f5cdc700 1 -- 172.21.15.43:0/1078162106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f00b06b0 con 0x7f85dc005200 2023-06-28T20:08:15.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.288+0000 7f85e7fff700 1 -- 172.21.15.43:0/1078162106 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e00055d0 con 0x7f85dc005200 2023-06-28T20:08:15.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.288+0000 7f85f5cdc700 1 -- 172.21.15.43:0/1078162106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f00b0c70 con 0x7f85dc005200 2023-06-28T20:08:15.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.289+0000 7f85f5cdc700 1 -- 172.21.15.43:0/1078162106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f00b0e30 con 0x7f85dc005200 2023-06-28T20:08:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.289+0000 7f85e7fff700 1 -- 172.21.15.43:0/1078162106 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f85e0003910 con 0x7f85dc005200 2023-06-28T20:08:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.290+0000 7f85e7fff700 1 --2- 172.21.15.43:0/1078162106 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f85d006ace0 0x7f85d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.290+0000 7f85e7fff700 1 -- 172.21.15.43:0/1078162106 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f85e0088f30 con 0x7f85dc005200 2023-06-28T20:08:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.290+0000 7f85eeffd700 1 --2- 172.21.15.43:0/1078162106 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f85d006ace0 0x7f85d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.290+0000 7f85eeffd700 1 --2- 172.21.15.43:0/1078162106 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f85d006ace0 0x7f85d006d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f85f01f3150 tx=0x7f85d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:15.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.294+0000 7f85e7fff700 1 -- 172.21.15.43:0/1078162106 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f85e0057dc0 con 0x7f85dc005200 2023-06-28T20:08:15.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.571+0000 7f85f5cdc700 1 -- 172.21.15.43:0/1078162106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f85f00728b0 con 0x7f85dc005200 2023-06-28T20:08:15.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.575+0000 7f85e7fff700 1 -- 172.21.15.43:0/1078162106 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f85e0057950 con 0x7f85dc005200 2023-06-28T20:08:15.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.577+0000 7f85f5cdc700 1 -- 172.21.15.43:0/1078162106 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f85d006ace0 msgr2=0x7f85d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:15.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.577+0000 7f85f5cdc700 1 --2- 172.21.15.43:0/1078162106 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f85d006ace0 0x7f85d006d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f85f01f3150 tx=0x7f85d8009040 comp rx=0 tx=0).stop 2023-06-28T20:08:15.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.577+0000 7f85f5cdc700 1 -- 172.21.15.43:0/1078162106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85dc005200 msgr2=0x7f85f00c70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:15.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.577+0000 7f85f5cdc700 1 --2- 172.21.15.43:0/1078162106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85dc005200 0x7f85f00c70c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f85e000f820 tx=0x7f85e000fbe0 comp rx=0 tx=0).stop 2023-06-28T20:08:15.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.577+0000 7f85f5cdc700 1 -- 172.21.15.43:0/1078162106 shutdown_connections 2023-06-28T20:08:15.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.577+0000 7f85f5cdc700 1 --2- 172.21.15.43:0/1078162106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f85f01e3c90 0x7f85f01e4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:15.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.577+0000 7f85f5cdc700 1 --2- 172.21.15.43:0/1078162106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85dc005200 0x7f85f00c70c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:15.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.577+0000 7f85f5cdc700 1 --2- 172.21.15.43:0/1078162106 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f85d006ace0 0x7f85d006d190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:15.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.577+0000 7f85f5cdc700 1 --2- 172.21.15.43:0/1078162106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f85f00b3240 0x7f85f00c4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:15.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.577+0000 7f85f5cdc700 1 -- 172.21.15.43:0/1078162106 >> 172.21.15.43:0/1078162106 conn(0x7f85f00664b0 msgr2=0x7f85f01e0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:15.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.577+0000 7f85f5cdc700 1 -- 172.21.15.43:0/1078162106 shutdown_connections 2023-06-28T20:08:15.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.577+0000 7f85f5cdc700 1 -- 172.21.15.43:0/1078162106 wait complete. 2023-06-28T20:08:15.594 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-06-28T20:08:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.716+0000 7fa3fa843700 1 Processor -- start 2023-06-28T20:08:15.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.716+0000 7fa3fa843700 1 -- start start 2023-06-28T20:08:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.717+0000 7fa3fa843700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3f40ceee0 0x7fa3f406d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.717+0000 7fa3fa843700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa3f40afdf0 0x7fa3f406d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.717+0000 7fa3fa843700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f40ae950 con 0x7fa3f415a5f0 2023-06-28T20:08:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.717+0000 7fa3fa843700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f415a9c0 con 0x7fa3f40afdf0 2023-06-28T20:08:15.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.717+0000 7fa3fa843700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f40add40 con 0x7fa3f40ceee0 2023-06-28T20:08:15.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.717+0000 7fa3f8de0700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa3f40afdf0 0x7fa3f406d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:15.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.717+0000 7fa3f3fff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa3f415a5f0 0x7fa3f40ce7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43994/0 (socket says 172.21.15.43:43994) 2023-06-28T20:08:15.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.717+0000 7fa3f3fff700 1 -- 172.21.15.43:0/559794388 learned_addr learned my addr 172.21.15.43:0/559794388 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:15.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.718+0000 7fa3f37fe700 1 --2- 172.21.15.43:0/559794388 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3f40ceee0 0x7fa3f406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:15.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.718+0000 7fa3f8de0700 1 -- 172.21.15.43:0/559794388 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3f40ceee0 msgr2=0x7fa3f406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:15.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.718+0000 7fa3f8de0700 1 --2- 172.21.15.43:0/559794388 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3f40ceee0 0x7fa3f406d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:15.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.718+0000 7fa3f8de0700 1 -- 172.21.15.43:0/559794388 >> v1:172.21.15.43:6789/0 conn(0x7fa3f415a5f0 legacy=0x7fa3f40ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:15.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.718+0000 7fa3f2ffd700 1 -- 172.21.15.43:0/559794388 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1209596660 0 0) 0x7fa3f40ae950 con 0x7fa3f415a5f0 2023-06-28T20:08:15.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.718+0000 7fa3f8de0700 1 -- 172.21.15.43:0/559794388 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f40ae020 con 0x7fa3f40afdf0 2023-06-28T20:08:15.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.718+0000 7fa3f8de0700 1 --2- 172.21.15.43:0/559794388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f40afdf0 0x7fa3f406d850 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec009d90 tx=0x7fa3ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7dc459c1d7f1a4 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:15.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.718+0000 7fa3f2ffd700 1 -- 172.21.15.43:0/559794388 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec00e070 con 0x7fa3f40afdf0 2023-06-28T20:08:15.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.718+0000 7fa3f2ffd700 1 -- 172.21.15.43:0/559794388 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ec00bbd0 con 0x7fa3f40afdf0 2023-06-28T20:08:15.725 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.718+0000 7fa3f2ffd700 1 -- 172.21.15.43:0/559794388 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec012430 con 0x7fa3f40afdf0 2023-06-28T20:08:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.719+0000 7fa3fa843700 1 -- 172.21.15.43:0/559794388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f40afdf0 msgr2=0x7fa3f406d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.719+0000 7fa3fa843700 1 --2- 172.21.15.43:0/559794388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f40afdf0 0x7fa3f406d850 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec009d90 tx=0x7fa3ec00b330 comp rx=0 tx=0).stop 2023-06-28T20:08:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.719+0000 7fa3fa843700 1 -- 172.21.15.43:0/559794388 shutdown_connections 2023-06-28T20:08:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.719+0000 7fa3fa843700 1 --2- 172.21.15.43:0/559794388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f40afdf0 0x7fa3f406d850 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.719+0000 7fa3fa843700 1 --2- 172.21.15.43:0/559794388 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3f40ceee0 0x7fa3f406d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.719+0000 7fa3fa843700 1 -- 172.21.15.43:0/559794388 >> 172.21.15.43:0/559794388 conn(0x7fa3f40664b0 msgr2=0x7fa3f40d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.719+0000 7fa3fa843700 1 -- 172.21.15.43:0/559794388 shutdown_connections 2023-06-28T20:08:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.719+0000 7fa3fa843700 1 -- 172.21.15.43:0/559794388 wait complete. 2023-06-28T20:08:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.720+0000 7fa3fa843700 1 Processor -- start 2023-06-28T20:08:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.720+0000 7fa3fa843700 1 -- start start 2023-06-28T20:08:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.721+0000 7fa3fa843700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3f40afdf0 0x7fa3f40ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:15.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.721+0000 7fa3fa843700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f40ceee0 0x7fa3f41e3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.721+0000 7fa3fa843700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3f415a5f0 0x7fa3f41e41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.721+0000 7fa3fa843700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f40cd6e0 con 0x7fa3f40ceee0 2023-06-28T20:08:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.721+0000 7fa3fa843700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f40cd9c0 con 0x7fa3f40afdf0 2023-06-28T20:08:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.721+0000 7fa3fa843700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f40cdca0 con 0x7fa3f415a5f0 2023-06-28T20:08:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.721+0000 7fa3f37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f40ceee0 0x7fa3f41e3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.721+0000 7fa3f37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f40ceee0 0x7fa3f41e3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54816/0 (socket says 172.21.15.43:54816) 2023-06-28T20:08:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.721+0000 7fa3f37fe700 1 -- 172.21.15.43:0/182197353 learned_addr learned my addr 172.21.15.43:0/182197353 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.721+0000 7fa3f3fff700 1 --2- 172.21.15.43:0/182197353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3f40afdf0 0x7fa3f40ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:15.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.721+0000 7fa3f8de0700 1 --2- 172.21.15.43:0/182197353 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3f415a5f0 0x7fa3f41e41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:15.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.721+0000 7fa3f37fe700 1 -- 172.21.15.43:0/182197353 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3f415a5f0 msgr2=0x7fa3f41e41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:15.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.721+0000 7fa3f37fe700 1 --2- 172.21.15.43:0/182197353 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3f415a5f0 0x7fa3f41e41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:15.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.721+0000 7fa3f37fe700 1 -- 172.21.15.43:0/182197353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3f40afdf0 msgr2=0x7fa3f40ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:15.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.721+0000 7fa3f37fe700 1 --2- 172.21.15.43:0/182197353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3f40afdf0 0x7fa3f40ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:15.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.721+0000 7fa3f37fe700 1 -- 172.21.15.43:0/182197353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ec009a70 con 0x7fa3f40ceee0 2023-06-28T20:08:15.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.721+0000 7fa3f3fff700 1 --2- 172.21.15.43:0/182197353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3f40afdf0 0x7fa3f40ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:15.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.721+0000 7fa3f37fe700 1 --2- 172.21.15.43:0/182197353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f40ceee0 0x7fa3f41e3660 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00e810 tx=0x7fa3dc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:15.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.721+0000 7fa3f0ff9700 1 -- 172.21.15.43:0/182197353 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc013610 con 0x7fa3f40ceee0 2023-06-28T20:08:15.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.722+0000 7fa3f0ff9700 1 -- 172.21.15.43:0/182197353 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3dc013770 con 0x7fa3f40ceee0 2023-06-28T20:08:15.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.722+0000 7fa3fa843700 1 -- 172.21.15.43:0/182197353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f40cdf80 con 0x7fa3f40ceee0 2023-06-28T20:08:15.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.722+0000 7fa3f0ff9700 1 -- 172.21.15.43:0/182197353 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc013b00 con 0x7fa3f40ceee0 2023-06-28T20:08:15.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.722+0000 7fa3fa843700 1 -- 172.21.15.43:0/182197353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f40cd320 con 0x7fa3f40ceee0 2023-06-28T20:08:15.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.723+0000 7fa3fa843700 1 -- 172.21.15.43:0/182197353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f40b1b70 con 0x7fa3f40ceee0 2023-06-28T20:08:15.734 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.723+0000 7fa3f0ff9700 1 -- 172.21.15.43:0/182197353 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa3dc013c60 con 0x7fa3f40ceee0 2023-06-28T20:08:15.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.723+0000 7fa3f0ff9700 1 --2- 172.21.15.43:0/182197353 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa3d406ace0 0x7fa3d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:15.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.724+0000 7fa3f0ff9700 1 -- 172.21.15.43:0/182197353 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa3dc088e10 con 0x7fa3f40ceee0 2023-06-28T20:08:15.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.724+0000 7fa3f3fff700 1 --2- 172.21.15.43:0/182197353 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa3d406ace0 0x7fa3d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:15.735 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.724+0000 7fa3f3fff700 1 --2- 172.21.15.43:0/182197353 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa3d406ace0 0x7fa3d406d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4001730 tx=0x7fa3e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:15.736 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:15.728+0000 7fa3f0ff9700 1 -- 172.21.15.43:0/182197353 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa3dc057ca0 con 0x7fa3f40ceee0 2023-06-28T20:08:16.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.008+0000 7fa3fa843700 1 -- 172.21.15.43:0/182197353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fa3f40b1970 con 0x7fa3f40ceee0 2023-06-28T20:08:16.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.011+0000 7fa3f0ff9700 1 -- 172.21.15.43:0/182197353 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa3dc0170a0 con 0x7fa3f40ceee0 2023-06-28T20:08:16.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.013+0000 7fa3fa843700 1 -- 172.21.15.43:0/182197353 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa3d406ace0 msgr2=0x7fa3d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:16.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.013+0000 7fa3fa843700 1 --2- 172.21.15.43:0/182197353 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa3d406ace0 0x7fa3d406d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4001730 tx=0x7fa3e4009040 comp rx=0 tx=0).stop 2023-06-28T20:08:16.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.013+0000 7fa3fa843700 1 -- 172.21.15.43:0/182197353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f40ceee0 msgr2=0x7fa3f41e3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:16.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.013+0000 7fa3fa843700 1 --2- 172.21.15.43:0/182197353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f40ceee0 0x7fa3f41e3660 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00e810 tx=0x7fa3dc00ebd0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.013+0000 7fa3fa843700 1 -- 172.21.15.43:0/182197353 shutdown_connections 2023-06-28T20:08:16.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.013+0000 7fa3fa843700 1 --2- 172.21.15.43:0/182197353 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa3d406ace0 0x7fa3d406d190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.013+0000 7fa3fa843700 1 --2- 172.21.15.43:0/182197353 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa3f415a5f0 0x7fa3f41e41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.013+0000 7fa3fa843700 1 --2- 172.21.15.43:0/182197353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3f40afdf0 0x7fa3f40ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.013+0000 7fa3fa843700 1 --2- 172.21.15.43:0/182197353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f40ceee0 0x7fa3f41e3660 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.013+0000 7fa3fa843700 1 -- 172.21.15.43:0/182197353 >> 172.21.15.43:0/182197353 conn(0x7fa3f40664b0 msgr2=0x7fa3f40d0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:16.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.013+0000 7fa3fa843700 1 -- 172.21.15.43:0/182197353 shutdown_connections 2023-06-28T20:08:16.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.013+0000 7fa3fa843700 1 -- 172.21.15.43:0/182197353 wait complete. 2023-06-28T20:08:16.031 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:08:16.156 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.152+0000 7f572f228700 1 Processor -- start 2023-06-28T20:08:16.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.152+0000 7f572f228700 1 -- start start 2023-06-28T20:08:16.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.153+0000 7f572f228700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5728070cb0 0x7f5728069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:16.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.153+0000 7f572f228700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f572806a850 con 0x7f572815d9f0 2023-06-28T20:08:16.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.153+0000 7f572f228700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5728071170 con 0x7f572806a430 2023-06-28T20:08:16.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.153+0000 7f572f228700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f572815ddc0 con 0x7f5728070cb0 2023-06-28T20:08:16.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.153+0000 7f572cfc4700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f572815d9f0 0x7f572806d3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44008/0 (socket says 172.21.15.43:44008) 2023-06-28T20:08:16.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.153+0000 7f572cfc4700 1 -- 172.21.15.43:0/842120683 learned_addr learned my addr 172.21.15.43:0/842120683 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:16.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.153+0000 7f5727fff700 1 --2- 172.21.15.43:0/842120683 >> v2:172.21.15.178:3300/0 conn(0x7f5728070cb0 0x7f5728069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:16.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.153+0000 7f57277fe700 1 -- 172.21.15.43:0/842120683 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3936851628 0 0) 0x7f572806a850 con 0x7f572815d9f0 2023-06-28T20:08:16.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.154+0000 7f57277fe700 1 -- 172.21.15.43:0/842120683 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57180035b0 con 0x7f572815d9f0 2023-06-28T20:08:16.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.154+0000 7f57277fe700 1 -- 172.21.15.43:0/842120683 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2840096629 0 0) 0x7f57180035b0 con 0x7f572815d9f0 2023-06-28T20:08:16.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.154+0000 7f57277fe700 1 -- 172.21.15.43:0/842120683 >> v2:172.21.15.178:3300/0 conn(0x7f5728070cb0 msgr2=0x7f5728069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:16.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.154+0000 7f57277fe700 1 --2- 172.21.15.43:0/842120683 >> v2:172.21.15.178:3300/0 conn(0x7f5728070cb0 0x7f5728069ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.154+0000 7f57277fe700 1 -- 172.21.15.43:0/842120683 >> v1:172.21.15.178:6789/0 conn(0x7f572806a430 legacy=0x7f57280cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:16.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.154+0000 7f57277fe700 1 -- 172.21.15.43:0/842120683 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5728071450 con 0x7f572815d9f0 2023-06-28T20:08:16.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.154+0000 7f57277fe700 1 -- 172.21.15.43:0/842120683 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f571c001670 con 0x7f572815d9f0 2023-06-28T20:08:16.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.154+0000 7f57277fe700 1 -- 172.21.15.43:0/842120683 >> v1:172.21.15.43:6789/0 conn(0x7f572815d9f0 legacy=0x7f572806d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:16.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.154+0000 7f57277fe700 1 --2- 172.21.15.43:0/842120683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5718003a00 0x7f5718003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:16.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.154+0000 7f57277fe700 1 --2- 172.21.15.43:0/842120683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5718004850 0x7f5718004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:16.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.154+0000 7f57277fe700 1 --2- 172.21.15.43:0/842120683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5718005200 0x7f571800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:16.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.154+0000 7f57277fe700 1 -- 172.21.15.43:0/842120683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572815ddc0 con 0x7f5718004850 2023-06-28T20:08:16.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.154+0000 7f57277fe700 1 -- 172.21.15.43:0/842120683 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571800c110 con 0x7f5718005200 2023-06-28T20:08:16.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.154+0000 7f57277fe700 1 -- 172.21.15.43:0/842120683 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571800c2d0 con 0x7f5718003a00 2023-06-28T20:08:16.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.154+0000 7f5727fff700 1 --2- 172.21.15.43:0/842120683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5718004850 0x7f5718004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:16.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.154+0000 7f572cfc4700 1 --2- 172.21.15.43:0/842120683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5718003a00 0x7f5718003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:16.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.154+0000 7f572d7c5700 1 --2- 172.21.15.43:0/842120683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5718005200 0x7f571800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:16.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.154+0000 7f5727fff700 1 -- 172.21.15.43:0/842120683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5718003a00 msgr2=0x7f5718003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:16.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.154+0000 7f5727fff700 1 --2- 172.21.15.43:0/842120683 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5718003a00 0x7f5718003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.154+0000 7f5727fff700 1 -- 172.21.15.43:0/842120683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5718005200 msgr2=0x7f571800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:16.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.154+0000 7f5727fff700 1 --2- 172.21.15.43:0/842120683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5718005200 0x7f571800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.154+0000 7f5727fff700 1 -- 172.21.15.43:0/842120683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57180035b0 con 0x7f5718004850 2023-06-28T20:08:16.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.154+0000 7f572d7c5700 1 -- 172.21.15.43:0/842120683 reap_dead start 2023-06-28T20:08:16.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.155+0000 7f5727fff700 1 --2- 172.21.15.43:0/842120683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5718004850 0x7f5718004cc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5714009b30 tx=0x7f5718005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c379393209527773 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:16.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.155+0000 7f57277fe700 1 -- 172.21.15.43:0/842120683 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571400e6c0 con 0x7f5718004850 2023-06-28T20:08:16.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.155+0000 7f57277fe700 1 -- 172.21.15.43:0/842120683 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571400e820 con 0x7f5718004850 2023-06-28T20:08:16.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.155+0000 7f57277fe700 1 -- 172.21.15.43:0/842120683 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571400eb90 con 0x7f5718004850 2023-06-28T20:08:16.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.155+0000 7f572f228700 1 -- 172.21.15.43:0/842120683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5718004850 msgr2=0x7f5718004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:16.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.155+0000 7f572f228700 1 --2- 172.21.15.43:0/842120683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5718004850 0x7f5718004cc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5714009b30 tx=0x7f5718005880 comp rx=0 tx=0).stop 2023-06-28T20:08:16.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.155+0000 7f572f228700 1 -- 172.21.15.43:0/842120683 shutdown_connections 2023-06-28T20:08:16.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.155+0000 7f572f228700 1 --2- 172.21.15.43:0/842120683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5718004850 0x7f5718004cc0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.155+0000 7f572f228700 1 -- 172.21.15.43:0/842120683 >> 172.21.15.43:0/842120683 conn(0x7f57280b1720 msgr2=0x7f57280b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:16.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.155+0000 7f572f228700 1 -- 172.21.15.43:0/842120683 shutdown_connections 2023-06-28T20:08:16.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.155+0000 7f572f228700 1 -- 172.21.15.43:0/842120683 wait complete. 2023-06-28T20:08:16.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.156+0000 7f572f228700 1 Processor -- start 2023-06-28T20:08:16.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.157+0000 7f572f228700 1 -- start start 2023-06-28T20:08:16.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.157+0000 7f572f228700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5718004850 0x7f57281f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:16.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.157+0000 7f572f228700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57280aed60 0x7f57281582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:16.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.157+0000 7f572f228700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57281f29a0 0x7f57281f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:16.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.157+0000 7f572f228700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57280af4d0 con 0x7f5718004850 2023-06-28T20:08:16.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.157+0000 7f572f228700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57281f3380 con 0x7f57281f29a0 2023-06-28T20:08:16.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.157+0000 7f572f228700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57281f24c0 con 0x7f57280aed60 2023-06-28T20:08:16.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.157+0000 7f572cfc4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5718004850 0x7f57281f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:16.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.157+0000 7f572cfc4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5718004850 0x7f57281f64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54834/0 (socket says 172.21.15.43:54834) 2023-06-28T20:08:16.168 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.158+0000 7f572cfc4700 1 -- 172.21.15.43:0/3208567771 learned_addr learned my addr 172.21.15.43:0/3208567771 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:16.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.158+0000 7f5727fff700 1 --2- 172.21.15.43:0/3208567771 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57280aed60 0x7f57281582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:16.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.158+0000 7f572d7c5700 1 --2- 172.21.15.43:0/3208567771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57281f29a0 0x7f57281f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:16.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.158+0000 7f572cfc4700 1 -- 172.21.15.43:0/3208567771 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57280aed60 msgr2=0x7f57281582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:16.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.158+0000 7f572cfc4700 1 --2- 172.21.15.43:0/3208567771 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57280aed60 0x7f57281582c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.158+0000 7f572cfc4700 1 -- 172.21.15.43:0/3208567771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57281f29a0 msgr2=0x7f57281f2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:16.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.158+0000 7f572cfc4700 1 --2- 172.21.15.43:0/3208567771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57281f29a0 0x7f57281f2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.158+0000 7f572cfc4700 1 -- 172.21.15.43:0/3208567771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57280ae520 con 0x7f5718004850 2023-06-28T20:08:16.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.158+0000 7f572cfc4700 1 --2- 172.21.15.43:0/3208567771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5718004850 0x7f57281f64d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f571c0072c0 tx=0x7f571c007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:16.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.158+0000 7f57257fa700 1 -- 172.21.15.43:0/3208567771 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571c00f8d0 con 0x7f5718004850 2023-06-28T20:08:16.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.158+0000 7f57257fa700 1 -- 172.21.15.43:0/3208567771 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571c00fa70 con 0x7f5718004850 2023-06-28T20:08:16.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.158+0000 7f572f228700 1 -- 172.21.15.43:0/3208567771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57280ae800 con 0x7f5718004850 2023-06-28T20:08:16.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.158+0000 7f57257fa700 1 -- 172.21.15.43:0/3208567771 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571c00fe20 con 0x7f5718004850 2023-06-28T20:08:16.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.158+0000 7f572f228700 1 -- 172.21.15.43:0/3208567771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57280af1d0 con 0x7f5718004850 2023-06-28T20:08:16.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.159+0000 7f572f228700 1 -- 172.21.15.43:0/3208567771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57281f2680 con 0x7f5718004850 2023-06-28T20:08:16.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.160+0000 7f57257fa700 1 -- 172.21.15.43:0/3208567771 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f571c01d460 con 0x7f5718004850 2023-06-28T20:08:16.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.160+0000 7f57257fa700 1 --2- 172.21.15.43:0/3208567771 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f570c06ace0 0x7f570c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:16.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.160+0000 7f57257fa700 1 -- 172.21.15.43:0/3208567771 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f571c0891b0 con 0x7f5718004850 2023-06-28T20:08:16.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.160+0000 7f5727fff700 1 --2- 172.21.15.43:0/3208567771 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f570c06ace0 0x7f570c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:16.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.160+0000 7f5727fff700 1 --2- 172.21.15.43:0/3208567771 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f570c06ace0 0x7f570c06d190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f57140090b0 tx=0x7f5714008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:16.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.164+0000 7f57257fa700 1 -- 172.21.15.43:0/3208567771 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f571c0545b0 con 0x7f5718004850 2023-06-28T20:08:16.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.364+0000 7f572f228700 1 -- 172.21.15.43:0/3208567771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57280728b0 con 0x7f5718004850 2023-06-28T20:08:16.369 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.365+0000 7f57257fa700 1 -- 172.21.15.43:0/3208567771 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+14581 (secure 0 0 0) 0x7f571c057bd0 con 0x7f5718004850 2023-06-28T20:08:16.369 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:16.369 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":31,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:08:12.930899+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:08:16.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.367+0000 7f572f228700 1 -- 172.21.15.43:0/3208567771 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f570c06ace0 msgr2=0x7f570c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:16.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.367+0000 7f572f228700 1 --2- 172.21.15.43:0/3208567771 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f570c06ace0 0x7f570c06d190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f57140090b0 tx=0x7f5714008030 comp rx=0 tx=0).stop 2023-06-28T20:08:16.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.367+0000 7f572f228700 1 -- 172.21.15.43:0/3208567771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5718004850 msgr2=0x7f57281f64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:16.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.367+0000 7f572f228700 1 --2- 172.21.15.43:0/3208567771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5718004850 0x7f57281f64d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f571c0072c0 tx=0x7f571c007680 comp rx=0 tx=0).stop 2023-06-28T20:08:16.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.367+0000 7f572f228700 1 -- 172.21.15.43:0/3208567771 shutdown_connections 2023-06-28T20:08:16.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.367+0000 7f572f228700 1 --2- 172.21.15.43:0/3208567771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57281f29a0 0x7f57281f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.367+0000 7f572f228700 1 --2- 172.21.15.43:0/3208567771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5718004850 0x7f57281f64d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.367+0000 7f572f228700 1 --2- 172.21.15.43:0/3208567771 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f570c06ace0 0x7f570c06d190 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.367+0000 7f572f228700 1 --2- 172.21.15.43:0/3208567771 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f57280aed60 0x7f57281582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.367+0000 7f572f228700 1 -- 172.21.15.43:0/3208567771 >> 172.21.15.43:0/3208567771 conn(0x7f57280b1720 msgr2=0x7f57280cb890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:16.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.367+0000 7f572f228700 1 -- 172.21.15.43:0/3208567771 shutdown_connections 2023-06-28T20:08:16.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.367+0000 7f572f228700 1 -- 172.21.15.43:0/3208567771 wait complete. 2023-06-28T20:08:16.384 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-28T20:07:13.580969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-28T20:07:14.038358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-28T20:08:16.385 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-28T20:08:16.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.508+0000 7fd170114700 1 Processor -- start 2023-06-28T20:08:16.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.509+0000 7fd170114700 1 -- start start 2023-06-28T20:08:16.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.509+0000 7fd170114700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd1680aec90 0x7fd168160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:16.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.509+0000 7fd170114700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd16806ccc0 0x7fd1680b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:16.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.509+0000 7fd170114700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd168160250 con 0x7fd1680af060 2023-06-28T20:08:16.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.509+0000 7fd170114700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd16815cb30 con 0x7fd1680aec90 2023-06-28T20:08:16.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.509+0000 7fd170114700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd168071410 con 0x7fd16806ccc0 2023-06-28T20:08:16.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.510+0000 7fd16deb0700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd1680aec90 0x7fd168160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:16.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.510+0000 7fd16deb0700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd1680aec90 0x7fd168160cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:40906/0 (socket says 172.21.15.43:40906) 2023-06-28T20:08:16.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.510+0000 7fd16deb0700 1 -- 172.21.15.43:0/1770658905 learned_addr learned my addr 172.21.15.43:0/1770658905 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:16.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.510+0000 7fd16e6b1700 1 --2- 172.21.15.43:0/1770658905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd16806ccc0 0x7fd1680b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:16.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.510+0000 7fd16deb0700 1 -- 172.21.15.43:0/1770658905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd16806ccc0 msgr2=0x7fd1680b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:16.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.510+0000 7fd16deb0700 1 --2- 172.21.15.43:0/1770658905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd16806ccc0 0x7fd1680b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.510+0000 7fd16deb0700 1 -- 172.21.15.43:0/1770658905 >> v1:172.21.15.178:6789/0 conn(0x7fd1680af060 legacy=0x7fd168070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:16.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.510+0000 7fd16deb0700 1 -- 172.21.15.43:0/1770658905 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1681604f0 con 0x7fd1680aec90 2023-06-28T20:08:16.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.510+0000 7fd16deb0700 1 --2- 172.21.15.43:0/1770658905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1680aec90 0x7fd168160cf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd1640098f0 tx=0x7fd164009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3db10fe670524eb5 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:16.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.510+0000 7fd16ceae700 1 -- 172.21.15.43:0/1770658905 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd164011830 con 0x7fd1680aec90 2023-06-28T20:08:16.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.510+0000 7fd16ceae700 1 -- 172.21.15.43:0/1770658905 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd164011990 con 0x7fd1680aec90 2023-06-28T20:08:16.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.510+0000 7fd16ceae700 1 -- 172.21.15.43:0/1770658905 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd164011d20 con 0x7fd1680aec90 2023-06-28T20:08:16.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.511+0000 7fd170114700 1 -- 172.21.15.43:0/1770658905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1680aec90 msgr2=0x7fd168160cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:16.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.511+0000 7fd170114700 1 --2- 172.21.15.43:0/1770658905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1680aec90 0x7fd168160cf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd1640098f0 tx=0x7fd164009c00 comp rx=0 tx=0).stop 2023-06-28T20:08:16.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.511+0000 7fd170114700 1 -- 172.21.15.43:0/1770658905 shutdown_connections 2023-06-28T20:08:16.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.511+0000 7fd170114700 1 --2- 172.21.15.43:0/1770658905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1680aec90 0x7fd168160cf0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.511+0000 7fd170114700 1 --2- 172.21.15.43:0/1770658905 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd16806ccc0 0x7fd1680b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.511+0000 7fd170114700 1 -- 172.21.15.43:0/1770658905 >> 172.21.15.43:0/1770658905 conn(0x7fd16806b000 msgr2=0x7fd16806b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:16.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.511+0000 7fd170114700 1 -- 172.21.15.43:0/1770658905 shutdown_connections 2023-06-28T20:08:16.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.511+0000 7fd170114700 1 -- 172.21.15.43:0/1770658905 wait complete. 2023-06-28T20:08:16.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.512+0000 7fd170114700 1 Processor -- start 2023-06-28T20:08:16.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.512+0000 7fd170114700 1 -- start start 2023-06-28T20:08:16.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.513+0000 7fd170114700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd16806ccc0 0x7fd1680b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:16.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.513+0000 7fd170114700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1680aec90 0x7fd1681526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:16.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.513+0000 7fd170114700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1680af060 0x7fd1680b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:16.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.513+0000 7fd170114700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1680b48a0 con 0x7fd1680aec90 2023-06-28T20:08:16.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.513+0000 7fd170114700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1680b2e80 con 0x7fd16806ccc0 2023-06-28T20:08:16.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.513+0000 7fd170114700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1680b3120 con 0x7fd1680af060 2023-06-28T20:08:16.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.513+0000 7fd16d6af700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1680aec90 0x7fd1681526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:16.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.513+0000 7fd16d6af700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1680aec90 0x7fd1681526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54850/0 (socket says 172.21.15.43:54850) 2023-06-28T20:08:16.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.513+0000 7fd16d6af700 1 -- 172.21.15.43:0/4029533596 learned_addr learned my addr 172.21.15.43:0/4029533596 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:16.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.513+0000 7fd16deb0700 1 --2- 172.21.15.43:0/4029533596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd16806ccc0 0x7fd1680b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:16.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.513+0000 7fd16e6b1700 1 --2- 172.21.15.43:0/4029533596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1680af060 0x7fd1680b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:16.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.514+0000 7fd16d6af700 1 -- 172.21.15.43:0/4029533596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1680af060 msgr2=0x7fd1680b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:16.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.514+0000 7fd16d6af700 1 --2- 172.21.15.43:0/4029533596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1680af060 0x7fd1680b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.514+0000 7fd16d6af700 1 -- 172.21.15.43:0/4029533596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd16806ccc0 msgr2=0x7fd1680b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:16.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.514+0000 7fd16d6af700 1 --2- 172.21.15.43:0/4029533596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd16806ccc0 0x7fd1680b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.514+0000 7fd16d6af700 1 -- 172.21.15.43:0/4029533596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd164009550 con 0x7fd1680aec90 2023-06-28T20:08:16.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.514+0000 7fd16d6af700 1 --2- 172.21.15.43:0/4029533596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1680aec90 0x7fd1681526d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00c450 tx=0x7fd15c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:16.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.514+0000 7fd15a7fc700 1 -- 172.21.15.43:0/4029533596 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c0052d0 con 0x7fd1680aec90 2023-06-28T20:08:16.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.514+0000 7fd15a7fc700 1 -- 172.21.15.43:0/4029533596 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15c005430 con 0x7fd1680aec90 2023-06-28T20:08:16.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.514+0000 7fd170114700 1 -- 172.21.15.43:0/4029533596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1680b3400 con 0x7fd1680aec90 2023-06-28T20:08:16.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.514+0000 7fd15a7fc700 1 -- 172.21.15.43:0/4029533596 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c0038c0 con 0x7fd1680aec90 2023-06-28T20:08:16.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.514+0000 7fd170114700 1 -- 172.21.15.43:0/4029533596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd168152c10 con 0x7fd1680aec90 2023-06-28T20:08:16.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.515+0000 7fd170114700 1 -- 172.21.15.43:0/4029533596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1680b4d50 con 0x7fd1680aec90 2023-06-28T20:08:16.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.515+0000 7fd15a7fc700 1 -- 172.21.15.43:0/4029533596 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd15c003a20 con 0x7fd1680aec90 2023-06-28T20:08:16.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.516+0000 7fd15a7fc700 1 --2- 172.21.15.43:0/4029533596 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd14c06ace0 0x7fd14c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:16.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.516+0000 7fd15a7fc700 1 -- 172.21.15.43:0/4029533596 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd15c088430 con 0x7fd1680aec90 2023-06-28T20:08:16.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.516+0000 7fd16deb0700 1 --2- 172.21.15.43:0/4029533596 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd14c06ace0 0x7fd14c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:16.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.516+0000 7fd16deb0700 1 --2- 172.21.15.43:0/4029533596 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd14c06ace0 0x7fd14c06d190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd164009c40 tx=0x7fd164008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:16.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.520+0000 7fd15a7fc700 1 -- 172.21.15.43:0/4029533596 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd1680b4d50 con 0x7fd1680aec90 2023-06-28T20:08:16.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.720+0000 7fd170114700 1 -- 172.21.15.43:0/4029533596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd1680b4d50 con 0x7fd1680aec90 2023-06-28T20:08:16.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.720+0000 7fd15a7fc700 1 -- 172.21.15.43:0/4029533596 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v31) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd1680b4d50 con 0x7fd1680aec90 2023-06-28T20:08:16.725 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-06-28T20:08:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.722+0000 7fd170114700 1 -- 172.21.15.43:0/4029533596 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd14c06ace0 msgr2=0x7fd14c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:16.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.722+0000 7fd170114700 1 --2- 172.21.15.43:0/4029533596 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd14c06ace0 0x7fd14c06d190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd164009c40 tx=0x7fd164008030 comp rx=0 tx=0).stop 2023-06-28T20:08:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.722+0000 7fd170114700 1 -- 172.21.15.43:0/4029533596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1680aec90 msgr2=0x7fd1681526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.722+0000 7fd170114700 1 --2- 172.21.15.43:0/4029533596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1680aec90 0x7fd1681526d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00c450 tx=0x7fd15c00c760 comp rx=0 tx=0).stop 2023-06-28T20:08:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.722+0000 7fd170114700 1 -- 172.21.15.43:0/4029533596 shutdown_connections 2023-06-28T20:08:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.722+0000 7fd170114700 1 --2- 172.21.15.43:0/4029533596 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd14c06ace0 0x7fd14c06d190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.722+0000 7fd170114700 1 --2- 172.21.15.43:0/4029533596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd1680af060 0x7fd1680b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.722+0000 7fd170114700 1 --2- 172.21.15.43:0/4029533596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd16806ccc0 0x7fd1680b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.722+0000 7fd170114700 1 --2- 172.21.15.43:0/4029533596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1680aec90 0x7fd1681526d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.722+0000 7fd170114700 1 -- 172.21.15.43:0/4029533596 >> 172.21.15.43:0/4029533596 conn(0x7fd16806b000 msgr2=0x7fd16806d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.722+0000 7fd170114700 1 -- 172.21.15.43:0/4029533596 shutdown_connections 2023-06-28T20:08:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.722+0000 7fd170114700 1 -- 172.21.15.43:0/4029533596 wait complete. 2023-06-28T20:08:16.738 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-28T20:08:16.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.862+0000 7f773ae05700 1 Processor -- start 2023-06-28T20:08:16.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.862+0000 7f773ae05700 1 -- start start 2023-06-28T20:08:16.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.863+0000 7f773ae05700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f773406d310 0x7f773406cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:16.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.863+0000 7f773ae05700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f773415cae0 0x7f773406d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:16.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.863+0000 7f773ae05700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77340d0680 con 0x7f77340cfe00 2023-06-28T20:08:16.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.863+0000 7f773ae05700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f773406dc20 con 0x7f773406d310 2023-06-28T20:08:16.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.863+0000 7f773ae05700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f773406bb20 con 0x7f773415cae0 2023-06-28T20:08:16.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.863+0000 7f7738ba1700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f773406d310 0x7f773406cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:16.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.863+0000 7f7738ba1700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f773406d310 0x7f773406cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54856/0 (socket says 172.21.15.43:54856) 2023-06-28T20:08:16.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.863+0000 7f7738ba1700 1 -- 172.21.15.43:0/670158293 learned_addr learned my addr 172.21.15.43:0/670158293 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.864+0000 7f7733fff700 1 --2- 172.21.15.43:0/670158293 >> v2:172.21.15.178:3300/0 conn(0x7f773415cae0 0x7f773406d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.864+0000 7f7738ba1700 1 -- 172.21.15.43:0/670158293 >> v2:172.21.15.178:3300/0 conn(0x7f773415cae0 msgr2=0x7f773406d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.864+0000 7f7738ba1700 1 --2- 172.21.15.43:0/670158293 >> v2:172.21.15.178:3300/0 conn(0x7f773415cae0 0x7f773406d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.864+0000 7f7738ba1700 1 -- 172.21.15.43:0/670158293 >> v1:172.21.15.178:6789/0 conn(0x7f77340cfe00 legacy=0x7f77340ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.864+0000 7f7738ba1700 1 -- 172.21.15.43:0/670158293 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773406be00 con 0x7f773406d310 2023-06-28T20:08:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.864+0000 7f7738ba1700 1 --2- 172.21.15.43:0/670158293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773406d310 0x7f773406cbc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f77340b37c0 tx=0x7f7728009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49eab02675fbf10c server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.864+0000 7f77337fe700 1 -- 172.21.15.43:0/670158293 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7728005e30 con 0x7f773406d310 2023-06-28T20:08:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.864+0000 7f77337fe700 1 -- 172.21.15.43:0/670158293 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7728004030 con 0x7f773406d310 2023-06-28T20:08:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.864+0000 7f77337fe700 1 -- 172.21.15.43:0/670158293 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77280043e0 con 0x7f773406d310 2023-06-28T20:08:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.864+0000 7f773ae05700 1 -- 172.21.15.43:0/670158293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773406d310 msgr2=0x7f773406cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.864+0000 7f773ae05700 1 --2- 172.21.15.43:0/670158293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773406d310 0x7f773406cbc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f77340b37c0 tx=0x7f7728009bd0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.864+0000 7f773ae05700 1 -- 172.21.15.43:0/670158293 shutdown_connections 2023-06-28T20:08:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.865+0000 7f773ae05700 1 --2- 172.21.15.43:0/670158293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773406d310 0x7f773406cbc0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.865+0000 7f773ae05700 1 --2- 172.21.15.43:0/670158293 >> v2:172.21.15.178:3300/0 conn(0x7f773415cae0 0x7f773406d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.865+0000 7f773ae05700 1 -- 172.21.15.43:0/670158293 >> 172.21.15.43:0/670158293 conn(0x7f77340664b0 msgr2=0x7f77340d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.865+0000 7f773ae05700 1 -- 172.21.15.43:0/670158293 shutdown_connections 2023-06-28T20:08:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.865+0000 7f773ae05700 1 -- 172.21.15.43:0/670158293 wait complete. 2023-06-28T20:08:16.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.866+0000 7f773ae05700 1 Processor -- start 2023-06-28T20:08:16.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.866+0000 7f773ae05700 1 -- start start 2023-06-28T20:08:16.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.867+0000 7f773ae05700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f773406d310 0x7f77340cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:16.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.867+0000 7f773ae05700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77340cfe00 0x7f773414eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:16.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.867+0000 7f773ae05700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773415cae0 0x7f77340b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:16.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.867+0000 7f773ae05700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77340cb2b0 con 0x7f773415cae0 2023-06-28T20:08:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.867+0000 7f773ae05700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77340b44d0 con 0x7f77340cfe00 2023-06-28T20:08:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.867+0000 7f773ae05700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77340cb520 con 0x7f773406d310 2023-06-28T20:08:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.867+0000 7f77393a2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773415cae0 0x7f77340b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.867+0000 7f77393a2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773415cae0 0x7f77340b3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54864/0 (socket says 172.21.15.43:54864) 2023-06-28T20:08:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.867+0000 7f77393a2700 1 -- 172.21.15.43:0/318543045 learned_addr learned my addr 172.21.15.43:0/318543045 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.867+0000 7f7738ba1700 1 --2- 172.21.15.43:0/318543045 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f773406d310 0x7f77340cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.867+0000 7f7733fff700 1 --2- 172.21.15.43:0/318543045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77340cfe00 0x7f773414eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:16.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.867+0000 7f77393a2700 1 -- 172.21.15.43:0/318543045 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f773406d310 msgr2=0x7f77340cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:16.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.867+0000 7f77393a2700 1 --2- 172.21.15.43:0/318543045 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f773406d310 0x7f77340cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.867+0000 7f77393a2700 1 -- 172.21.15.43:0/318543045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77340cfe00 msgr2=0x7f773414eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:16.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.867+0000 7f77393a2700 1 --2- 172.21.15.43:0/318543045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77340cfe00 0x7f773414eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:16.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.867+0000 7f77393a2700 1 -- 172.21.15.43:0/318543045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7728009500 con 0x7f773415cae0 2023-06-28T20:08:16.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.867+0000 7f7733fff700 1 --2- 172.21.15.43:0/318543045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77340cfe00 0x7f773414eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:16.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.867+0000 7f77393a2700 1 --2- 172.21.15.43:0/318543045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773415cae0 0x7f77340b3a30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f772c00cc50 tx=0x7f772c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:16.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.867+0000 7f77317fa700 1 -- 172.21.15.43:0/318543045 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c00ad80 con 0x7f773415cae0 2023-06-28T20:08:16.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.867+0000 7f77317fa700 1 -- 172.21.15.43:0/318543045 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772c00ce50 con 0x7f773415cae0 2023-06-28T20:08:16.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.867+0000 7f773ae05700 1 -- 172.21.15.43:0/318543045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77340cb800 con 0x7f773415cae0 2023-06-28T20:08:16.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.867+0000 7f77317fa700 1 -- 172.21.15.43:0/318543045 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772c003c40 con 0x7f773415cae0 2023-06-28T20:08:16.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.867+0000 7f773ae05700 1 -- 172.21.15.43:0/318543045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77340b47b0 con 0x7f773415cae0 2023-06-28T20:08:16.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.868+0000 7f773ae05700 1 -- 172.21.15.43:0/318543045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77340728b0 con 0x7f773415cae0 2023-06-28T20:08:16.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.869+0000 7f77317fa700 1 -- 172.21.15.43:0/318543045 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f772c0050d0 con 0x7f773415cae0 2023-06-28T20:08:16.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.869+0000 7f77317fa700 1 --2- 172.21.15.43:0/318543045 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f771806ace0 0x7f771806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:16.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.869+0000 7f77317fa700 1 -- 172.21.15.43:0/318543045 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f772c089240 con 0x7f773415cae0 2023-06-28T20:08:16.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.869+0000 7f7738ba1700 1 --2- 172.21.15.43:0/318543045 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f771806ace0 0x7f771806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:16.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.870+0000 7f7738ba1700 1 --2- 172.21.15.43:0/318543045 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f771806ace0 0x7f771806d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f77340b37c0 tx=0x7f7728008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:16.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:16.874+0000 7f77317fa700 1 -- 172.21.15.43:0/318543045 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f772c054640 con 0x7f773415cae0 2023-06-28T20:08:17.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.070+0000 7f773ae05700 1 -- 172.21.15.43:0/318543045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f77340cb9f0 con 0x7f773415cae0 2023-06-28T20:08:17.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.070+0000 7f77317fa700 1 -- 172.21.15.43:0/318543045 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) v1 ==== 93+0+10 (secure 0 0 0) 0x7f772c057c60 con 0x7f773415cae0 2023-06-28T20:08:17.075 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-06-28T20:08:17.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.072+0000 7f773ae05700 1 -- 172.21.15.43:0/318543045 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f771806ace0 msgr2=0x7f771806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:17.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.072+0000 7f773ae05700 1 --2- 172.21.15.43:0/318543045 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f771806ace0 0x7f771806d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f77340b37c0 tx=0x7f7728008070 comp rx=0 tx=0).stop 2023-06-28T20:08:17.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.073+0000 7f773ae05700 1 -- 172.21.15.43:0/318543045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773415cae0 msgr2=0x7f77340b3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:17.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.073+0000 7f773ae05700 1 --2- 172.21.15.43:0/318543045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773415cae0 0x7f77340b3a30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f772c00cc50 tx=0x7f772c00a340 comp rx=0 tx=0).stop 2023-06-28T20:08:17.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.073+0000 7f773ae05700 1 -- 172.21.15.43:0/318543045 shutdown_connections 2023-06-28T20:08:17.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.073+0000 7f773ae05700 1 --2- 172.21.15.43:0/318543045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773415cae0 0x7f77340b3a30 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:17.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.073+0000 7f773ae05700 1 --2- 172.21.15.43:0/318543045 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f771806ace0 0x7f771806d190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:17.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.073+0000 7f773ae05700 1 --2- 172.21.15.43:0/318543045 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f773406d310 0x7f77340cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:17.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.073+0000 7f773ae05700 1 --2- 172.21.15.43:0/318543045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77340cfe00 0x7f773414eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:17.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.073+0000 7f773ae05700 1 -- 172.21.15.43:0/318543045 >> 172.21.15.43:0/318543045 conn(0x7f77340664b0 msgr2=0x7f77340c7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:17.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.073+0000 7f773ae05700 1 -- 172.21.15.43:0/318543045 shutdown_connections 2023-06-28T20:08:17.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.073+0000 7f773ae05700 1 -- 172.21.15.43:0/318543045 wait complete. 2023-06-28T20:08:17.089 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-06-28T20:08:17.214 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.210+0000 7f5f6679d700 1 Processor -- start 2023-06-28T20:08:17.216 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.210+0000 7f5f6679d700 1 -- start start 2023-06-28T20:08:17.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.211+0000 7f5f6679d700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5f6006a4a0 0x7f5f60142670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:17.218 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.211+0000 7f5f6679d700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f600ccf30 con 0x7f5f6006a0d0 2023-06-28T20:08:17.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.211+0000 7f5f6679d700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f600cd210 con 0x7f5f600cc460 2023-06-28T20:08:17.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.211+0000 7f5f6679d700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f600707e0 con 0x7f5f6006a4a0 2023-06-28T20:08:17.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.211+0000 7f5f6579b700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5f6006a0d0 0x7f5f600ca130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44054/0 (socket says 172.21.15.43:44054) 2023-06-28T20:08:17.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.211+0000 7f5f6579b700 1 -- 172.21.15.43:0/3967882228 learned_addr learned my addr 172.21.15.43:0/3967882228 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:17.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.211+0000 7f5f64f9a700 1 --2- 172.21.15.43:0/3967882228 >> v2:172.21.15.178:3300/0 conn(0x7f5f6006a4a0 0x7f5f60142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:17.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.211+0000 7f5f57fff700 1 -- 172.21.15.43:0/3967882228 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2313861629 0 0) 0x7f5f600ccf30 con 0x7f5f6006a0d0 2023-06-28T20:08:17.219 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.211+0000 7f5f57fff700 1 -- 172.21.15.43:0/3967882228 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f4c0035b0 con 0x7f5f6006a0d0 2023-06-28T20:08:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.212+0000 7f5f57fff700 1 -- 172.21.15.43:0/3967882228 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 134452895 0 0) 0x7f5f4c0035b0 con 0x7f5f6006a0d0 2023-06-28T20:08:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.212+0000 7f5f57fff700 1 -- 172.21.15.43:0/3967882228 >> v2:172.21.15.178:3300/0 conn(0x7f5f6006a4a0 msgr2=0x7f5f60142670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.212+0000 7f5f57fff700 1 --2- 172.21.15.43:0/3967882228 >> v2:172.21.15.178:3300/0 conn(0x7f5f6006a4a0 0x7f5f60142670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.212+0000 7f5f57fff700 1 -- 172.21.15.43:0/3967882228 >> v1:172.21.15.178:6789/0 conn(0x7f5f600cc460 legacy=0x7f5f600711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.212+0000 7f5f57fff700 1 -- 172.21.15.43:0/3967882228 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f600709f0 con 0x7f5f6006a0d0 2023-06-28T20:08:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.212+0000 7f5f57fff700 1 -- 172.21.15.43:0/3967882228 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f5f50001670 con 0x7f5f6006a0d0 2023-06-28T20:08:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.212+0000 7f5f57fff700 1 -- 172.21.15.43:0/3967882228 >> v1:172.21.15.43:6789/0 conn(0x7f5f6006a0d0 legacy=0x7f5f600ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.212+0000 7f5f57fff700 1 --2- 172.21.15.43:0/3967882228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f4c003a00 0x7f5f4c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.212+0000 7f5f57fff700 1 --2- 172.21.15.43:0/3967882228 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f4c004850 0x7f5f4c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.212+0000 7f5f57fff700 1 --2- 172.21.15.43:0/3967882228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f4c005200 0x7f5f4c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.212+0000 7f5f57fff700 1 -- 172.21.15.43:0/3967882228 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f600707e0 con 0x7f5f4c005200 2023-06-28T20:08:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.212+0000 7f5f57fff700 1 -- 172.21.15.43:0/3967882228 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c00c080 con 0x7f5f4c003a00 2023-06-28T20:08:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.212+0000 7f5f57fff700 1 -- 172.21.15.43:0/3967882228 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c00c240 con 0x7f5f4c004850 2023-06-28T20:08:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.212+0000 7f5f65f9c700 1 --2- 172.21.15.43:0/3967882228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f4c005200 0x7f5f4c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.212+0000 7f5f6579b700 1 --2- 172.21.15.43:0/3967882228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f4c003a00 0x7f5f4c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:17.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.212+0000 7f5f64f9a700 1 --2- 172.21.15.43:0/3967882228 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f4c004850 0x7f5f4c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:17.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.212+0000 7f5f65f9c700 1 -- 172.21.15.43:0/3967882228 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f4c004850 msgr2=0x7f5f4c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:17.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.212+0000 7f5f65f9c700 1 --2- 172.21.15.43:0/3967882228 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f4c004850 0x7f5f4c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:17.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.212+0000 7f5f65f9c700 1 -- 172.21.15.43:0/3967882228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f4c003a00 msgr2=0x7f5f4c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:17.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.212+0000 7f5f65f9c700 1 --2- 172.21.15.43:0/3967882228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f4c003a00 0x7f5f4c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:17.222 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.212+0000 7f5f65f9c700 1 -- 172.21.15.43:0/3967882228 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4c0035b0 con 0x7f5f4c005200 2023-06-28T20:08:17.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.212+0000 7f5f65f9c700 1 -- 172.21.15.43:0/3967882228 reap_dead start 2023-06-28T20:08:17.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.212+0000 7f5f65f9c700 1 --2- 172.21.15.43:0/3967882228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f4c005200 0x7f5f4c00ba10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800c190 tx=0x7f5f4c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae7a32d0f75f1ba server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:17.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.213+0000 7f5f57fff700 1 -- 172.21.15.43:0/3967882228 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f58001db0 con 0x7f5f4c005200 2023-06-28T20:08:17.223 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.213+0000 7f5f57fff700 1 -- 172.21.15.43:0/3967882228 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f5800ccd0 con 0x7f5f4c005200 2023-06-28T20:08:17.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.213+0000 7f5f57fff700 1 -- 172.21.15.43:0/3967882228 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f58003c40 con 0x7f5f4c005200 2023-06-28T20:08:17.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.213+0000 7f5f6679d700 1 -- 172.21.15.43:0/3967882228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f4c005200 msgr2=0x7f5f4c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:17.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.213+0000 7f5f6679d700 1 --2- 172.21.15.43:0/3967882228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f4c005200 0x7f5f4c00ba10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800c190 tx=0x7f5f4c005880 comp rx=0 tx=0).stop 2023-06-28T20:08:17.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.213+0000 7f5f6679d700 1 -- 172.21.15.43:0/3967882228 shutdown_connections 2023-06-28T20:08:17.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.213+0000 7f5f6679d700 1 --2- 172.21.15.43:0/3967882228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f4c005200 0x7f5f4c00ba10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:17.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.213+0000 7f5f6679d700 1 -- 172.21.15.43:0/3967882228 >> 172.21.15.43:0/3967882228 conn(0x7f5f600b1790 msgr2=0x7f5f600b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.213+0000 7f5f6679d700 1 -- 172.21.15.43:0/3967882228 shutdown_connections 2023-06-28T20:08:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.213+0000 7f5f6679d700 1 -- 172.21.15.43:0/3967882228 wait complete. 2023-06-28T20:08:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.215+0000 7f5f6679d700 1 Processor -- start 2023-06-28T20:08:17.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.215+0000 7f5f6679d700 1 -- start start 2023-06-28T20:08:17.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.216+0000 7f5f6679d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f4c005200 0x7f5f601f5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:17.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.216+0000 7f5f6679d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f601f5980 0x7f5f600ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:17.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.216+0000 7f5f6679d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f600ca8f0 0x7f5f6013dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:17.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.216+0000 7f5f6679d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f601f5df0 con 0x7f5f600ca8f0 2023-06-28T20:08:17.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.216+0000 7f5f6679d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f600b2b30 con 0x7f5f601f5980 2023-06-28T20:08:17.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.216+0000 7f5f6679d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f600cb100 con 0x7f5f4c005200 2023-06-28T20:08:17.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.216+0000 7f5f65f9c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f600ca8f0 0x7f5f6013dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:17.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.216+0000 7f5f65f9c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f600ca8f0 0x7f5f6013dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54884/0 (socket says 172.21.15.43:54884) 2023-06-28T20:08:17.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.216+0000 7f5f65f9c700 1 -- 172.21.15.43:0/2689986186 learned_addr learned my addr 172.21.15.43:0/2689986186 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.216+0000 7f5f6579b700 1 --2- 172.21.15.43:0/2689986186 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f4c005200 0x7f5f601f5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.216+0000 7f5f64f9a700 1 --2- 172.21.15.43:0/2689986186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f601f5980 0x7f5f600ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.216+0000 7f5f65f9c700 1 -- 172.21.15.43:0/2689986186 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f4c005200 msgr2=0x7f5f601f5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.216+0000 7f5f65f9c700 1 --2- 172.21.15.43:0/2689986186 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f4c005200 0x7f5f601f5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.216+0000 7f5f65f9c700 1 -- 172.21.15.43:0/2689986186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f601f5980 msgr2=0x7f5f600ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.216+0000 7f5f65f9c700 1 --2- 172.21.15.43:0/2689986186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f601f5980 0x7f5f600ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:17.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.216+0000 7f5f65f9c700 1 -- 172.21.15.43:0/2689986186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f600cb3e0 con 0x7f5f600ca8f0 2023-06-28T20:08:17.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.216+0000 7f5f65f9c700 1 --2- 172.21.15.43:0/2689986186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f600ca8f0 0x7f5f6013dc00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5f600b2560 tx=0x7f5f580014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:17.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.216+0000 7f5f55ffb700 1 -- 172.21.15.43:0/2689986186 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f58005070 con 0x7f5f600ca8f0 2023-06-28T20:08:17.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.216+0000 7f5f55ffb700 1 -- 172.21.15.43:0/2689986186 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f580051d0 con 0x7f5f600ca8f0 2023-06-28T20:08:17.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.217+0000 7f5f55ffb700 1 -- 172.21.15.43:0/2689986186 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f58005380 con 0x7f5f600ca8f0 2023-06-28T20:08:17.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.217+0000 7f5f6679d700 1 -- 172.21.15.43:0/2689986186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f600cb6c0 con 0x7f5f600ca8f0 2023-06-28T20:08:17.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.217+0000 7f5f6679d700 1 -- 172.21.15.43:0/2689986186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f600cad60 con 0x7f5f600ca8f0 2023-06-28T20:08:17.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.218+0000 7f5f6679d700 1 -- 172.21.15.43:0/2689986186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f600728b0 con 0x7f5f600ca8f0 2023-06-28T20:08:17.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.218+0000 7f5f55ffb700 1 -- 172.21.15.43:0/2689986186 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5f580054e0 con 0x7f5f600ca8f0 2023-06-28T20:08:17.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.218+0000 7f5f55ffb700 1 --2- 172.21.15.43:0/2689986186 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5f4406ace0 0x7f5f4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:17.234 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.219+0000 7f5f55ffb700 1 -- 172.21.15.43:0/2689986186 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5f58089c90 con 0x7f5f600ca8f0 2023-06-28T20:08:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.219+0000 7f5f6579b700 1 --2- 172.21.15.43:0/2689986186 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5f4406ace0 0x7f5f4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.219+0000 7f5f6579b700 1 --2- 172.21.15.43:0/2689986186 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5f4406ace0 0x7f5f4406d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5f50001eb0 tx=0x7f5f50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.223+0000 7f5f55ffb700 1 -- 172.21.15.43:0/2689986186 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5f58055080 con 0x7f5f600ca8f0 2023-06-28T20:08:17.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.424+0000 7f5f6679d700 1 -- 172.21.15.43:0/2689986186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5f60140810 con 0x7f5f600ca8f0 2023-06-28T20:08:17.429 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.425+0000 7f5f55ffb700 1 -- 172.21.15.43:0/2689986186 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+126 (secure 0 0 0) 0x7f5f580586a0 con 0x7f5f600ca8f0 2023-06-28T20:08:17.429 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:17.430 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2023-06-28T20:08:17.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:17.430 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2023-06-28T20:08:17.430 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": "rbd" 2023-06-28T20:08:17.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2023-06-28T20:08:17.430 INFO:teuthology.orchestra.run.smithi043.stdout: { 2023-06-28T20:08:17.430 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 2, 2023-06-28T20:08:17.431 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2023-06-28T20:08:17.431 INFO:teuthology.orchestra.run.smithi043.stdout: } 2023-06-28T20:08:17.431 INFO:teuthology.orchestra.run.smithi043.stdout:] 2023-06-28T20:08:17.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.427+0000 7f5f6679d700 1 -- 172.21.15.43:0/2689986186 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5f4406ace0 msgr2=0x7f5f4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:17.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.427+0000 7f5f6679d700 1 --2- 172.21.15.43:0/2689986186 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5f4406ace0 0x7f5f4406d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5f50001eb0 tx=0x7f5f50009040 comp rx=0 tx=0).stop 2023-06-28T20:08:17.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.427+0000 7f5f6679d700 1 -- 172.21.15.43:0/2689986186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f600ca8f0 msgr2=0x7f5f6013dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:17.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.427+0000 7f5f6679d700 1 --2- 172.21.15.43:0/2689986186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f600ca8f0 0x7f5f6013dc00 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5f600b2560 tx=0x7f5f580014a0 comp rx=0 tx=0).stop 2023-06-28T20:08:17.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.427+0000 7f5f6679d700 1 -- 172.21.15.43:0/2689986186 shutdown_connections 2023-06-28T20:08:17.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.427+0000 7f5f6679d700 1 --2- 172.21.15.43:0/2689986186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f600ca8f0 0x7f5f6013dc00 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:17.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.427+0000 7f5f6679d700 1 --2- 172.21.15.43:0/2689986186 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5f4406ace0 0x7f5f4406d190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:17.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.427+0000 7f5f6679d700 1 --2- 172.21.15.43:0/2689986186 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5f4c005200 0x7f5f601f5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:17.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.427+0000 7f5f6679d700 1 --2- 172.21.15.43:0/2689986186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f601f5980 0x7f5f600ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:17.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.427+0000 7f5f6679d700 1 -- 172.21.15.43:0/2689986186 >> 172.21.15.43:0/2689986186 conn(0x7f5f600b1790 msgr2=0x7f5f60071040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:17.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.427+0000 7f5f6679d700 1 -- 172.21.15.43:0/2689986186 shutdown_connections 2023-06-28T20:08:17.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.427+0000 7f5f6679d700 1 -- 172.21.15.43:0/2689986186 wait complete. 2023-06-28T20:08:17.443 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-06-28T20:08:17.444 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-06-28T20:08:17.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.569+0000 7fbfc277e700 1 Processor -- start 2023-06-28T20:08:17.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.569+0000 7fbfc277e700 1 -- start start 2023-06-28T20:08:17.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.570+0000 7fbfc277e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfbc15ceb0 0x7fbfbc15d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:17.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.570+0000 7fbfc277e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fbfbc0b4f20 0x7fbfbc0cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:17.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.570+0000 7fbfc277e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbfbc0d0e20 0x7fbfbc0d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:17.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.570+0000 7fbfc277e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfbc0b8960 con 0x7fbfbc0d0e20 2023-06-28T20:08:17.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.570+0000 7fbfc277e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfbc0ce770 con 0x7fbfbc0b4f20 2023-06-28T20:08:17.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.570+0000 7fbfc277e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc0cd410 con 0x7fbfbc15ceb0 2023-06-28T20:08:17.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.570+0000 7fbfc0d1b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbfbc0d0e20 0x7fbfbc0d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:17.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.570+0000 7fbfc0d1b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbfbc0d0e20 0x7fbfbc0d1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54892/0 (socket says 172.21.15.43:54892) 2023-06-28T20:08:17.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.570+0000 7fbfc0d1b700 1 -- 172.21.15.43:0/3671447362 learned_addr learned my addr 172.21.15.43:0/3671447362 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:17.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.570+0000 7fbfbb7fe700 1 --2- 172.21.15.43:0/3671447362 >> v2:172.21.15.178:3300/0 conn(0x7fbfbc0b4f20 0x7fbfbc0cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:17.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.570+0000 7fbfbbfff700 1 --2- 172.21.15.43:0/3671447362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfbc15ceb0 0x7fbfbc15d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:17.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.570+0000 7fbfc0d1b700 1 -- 172.21.15.43:0/3671447362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfbc15ceb0 msgr2=0x7fbfbc15d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:17.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.570+0000 7fbfc0d1b700 1 --2- 172.21.15.43:0/3671447362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfbc15ceb0 0x7fbfbc15d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:17.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.570+0000 7fbfc0d1b700 1 -- 172.21.15.43:0/3671447362 >> v2:172.21.15.178:3300/0 conn(0x7fbfbc0b4f20 msgr2=0x7fbfbc0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:17.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.570+0000 7fbfc0d1b700 1 --2- 172.21.15.43:0/3671447362 >> v2:172.21.15.178:3300/0 conn(0x7fbfbc0b4f20 0x7fbfbc0cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:17.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.570+0000 7fbfc0d1b700 1 -- 172.21.15.43:0/3671447362 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc0d0990 con 0x7fbfbc0d0e20 2023-06-28T20:08:17.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.570+0000 7fbfc0d1b700 1 --2- 172.21.15.43:0/3671447362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfbc0d0e20 0x7fbfbc0d1260 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc06b970 tx=0x7fbfb400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd98f560b3dc649 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:17.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.570+0000 7fbfbaffd700 1 -- 172.21.15.43:0/3671447362 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb400e070 con 0x7fbfbc0d0e20 2023-06-28T20:08:17.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.571+0000 7fbfbaffd700 1 -- 172.21.15.43:0/3671447362 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb400b910 con 0x7fbfbc0d0e20 2023-06-28T20:08:17.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.571+0000 7fbfbaffd700 1 -- 172.21.15.43:0/3671447362 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb400bcc0 con 0x7fbfbc0d0e20 2023-06-28T20:08:17.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.571+0000 7fbfc277e700 1 -- 172.21.15.43:0/3671447362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfbc0d0e20 msgr2=0x7fbfbc0d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:17.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.571+0000 7fbfc277e700 1 --2- 172.21.15.43:0/3671447362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfbc0d0e20 0x7fbfbc0d1260 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc06b970 tx=0x7fbfb400b330 comp rx=0 tx=0).stop 2023-06-28T20:08:17.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.571+0000 7fbfc277e700 1 -- 172.21.15.43:0/3671447362 shutdown_connections 2023-06-28T20:08:17.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.571+0000 7fbfc277e700 1 --2- 172.21.15.43:0/3671447362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfbc0d0e20 0x7fbfbc0d1260 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:17.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.571+0000 7fbfc277e700 1 --2- 172.21.15.43:0/3671447362 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfbc15ceb0 0x7fbfbc15d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:17.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.571+0000 7fbfc277e700 1 --2- 172.21.15.43:0/3671447362 >> v2:172.21.15.178:3300/0 conn(0x7fbfbc0b4f20 0x7fbfbc0cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:17.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.571+0000 7fbfc277e700 1 -- 172.21.15.43:0/3671447362 >> 172.21.15.43:0/3671447362 conn(0x7fbfbc06a5c0 msgr2=0x7fbfbc06a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:17.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.571+0000 7fbfc277e700 1 -- 172.21.15.43:0/3671447362 shutdown_connections 2023-06-28T20:08:17.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.571+0000 7fbfc277e700 1 -- 172.21.15.43:0/3671447362 wait complete. 2023-06-28T20:08:17.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.572+0000 7fbfc277e700 1 Processor -- start 2023-06-28T20:08:17.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.572+0000 7fbfc277e700 1 -- start start 2023-06-28T20:08:17.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.573+0000 7fbfc277e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfbc0b4f20 0x7fbfbc0cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:17.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.573+0000 7fbfc277e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfbc0d0e20 0x7fbfbc157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:17.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.573+0000 7fbfc277e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfbc15ceb0 0x7fbfbc158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:17.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.573+0000 7fbfc277e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc1594b0 con 0x7fbfbc15ceb0 2023-06-28T20:08:17.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.573+0000 7fbfc277e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc159790 con 0x7fbfbc0b4f20 2023-06-28T20:08:17.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.573+0000 7fbfc277e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc159a70 con 0x7fbfbc0d0e20 2023-06-28T20:08:17.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.573+0000 7fbfc0d1b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfbc15ceb0 0x7fbfbc158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:17.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.573+0000 7fbfc0d1b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfbc15ceb0 0x7fbfbc158df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54898/0 (socket says 172.21.15.43:54898) 2023-06-28T20:08:17.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.573+0000 7fbfc0d1b700 1 -- 172.21.15.43:0/1758468974 learned_addr learned my addr 172.21.15.43:0/1758468974 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:17.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.573+0000 7fbfbbfff700 1 --2- 172.21.15.43:0/1758468974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfbc0b4f20 0x7fbfbc0cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:17.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.573+0000 7fbfbb7fe700 1 --2- 172.21.15.43:0/1758468974 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfbc0d0e20 0x7fbfbc157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:17.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.574+0000 7fbfc0d1b700 1 -- 172.21.15.43:0/1758468974 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfbc0d0e20 msgr2=0x7fbfbc157570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:17.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.574+0000 7fbfc0d1b700 1 --2- 172.21.15.43:0/1758468974 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfbc0d0e20 0x7fbfbc157570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:17.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.574+0000 7fbfc0d1b700 1 -- 172.21.15.43:0/1758468974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfbc0b4f20 msgr2=0x7fbfbc0cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:17.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.574+0000 7fbfc0d1b700 1 --2- 172.21.15.43:0/1758468974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfbc0b4f20 0x7fbfbc0cb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:17.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.574+0000 7fbfc0d1b700 1 -- 172.21.15.43:0/1758468974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb4009b30 con 0x7fbfbc15ceb0 2023-06-28T20:08:17.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.574+0000 7fbfc0d1b700 1 --2- 172.21.15.43:0/1758468974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfbc15ceb0 0x7fbfbc158df0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc06b260 tx=0x7fbfb4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:17.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.574+0000 7fbfb8ff9700 1 -- 172.21.15.43:0/1758468974 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb40099b0 con 0x7fbfbc15ceb0 2023-06-28T20:08:17.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.574+0000 7fbfb8ff9700 1 -- 172.21.15.43:0/1758468974 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb400ce80 con 0x7fbfbc15ceb0 2023-06-28T20:08:17.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.574+0000 7fbfb8ff9700 1 -- 172.21.15.43:0/1758468974 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4004490 con 0x7fbfbc15ceb0 2023-06-28T20:08:17.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.574+0000 7fbfc277e700 1 -- 172.21.15.43:0/1758468974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfbc159d50 con 0x7fbfbc15ceb0 2023-06-28T20:08:17.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.574+0000 7fbfc277e700 1 -- 172.21.15.43:0/1758468974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfbc0b7be0 con 0x7fbfbc15ceb0 2023-06-28T20:08:17.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.575+0000 7fbfc277e700 1 -- 172.21.15.43:0/1758468974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfbc0ca0d0 con 0x7fbfbc15ceb0 2023-06-28T20:08:17.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.575+0000 7fbfb8ff9700 1 -- 172.21.15.43:0/1758468974 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbfb40045f0 con 0x7fbfbc15ceb0 2023-06-28T20:08:17.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.576+0000 7fbfb8ff9700 1 --2- 172.21.15.43:0/1758468974 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbf9c06ace0 0x7fbf9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:17.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.576+0000 7fbfb8ff9700 1 -- 172.21.15.43:0/1758468974 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fbfb408a3a0 con 0x7fbfbc15ceb0 2023-06-28T20:08:17.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.576+0000 7fbfbbfff700 1 --2- 172.21.15.43:0/1758468974 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbf9c06ace0 0x7fbf9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:17.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.576+0000 7fbfbbfff700 1 --2- 172.21.15.43:0/1758468974 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbf9c06ace0 0x7fbf9c06d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009b40 tx=0x7fbfac009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:17.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.580+0000 7fbfb8ff9700 1 -- 172.21.15.43:0/1758468974 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbfbc0ca0d0 con 0x7fbfbc15ceb0 2023-06-28T20:08:17.783 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:17.779+0000 7fbfc277e700 1 -- 172.21.15.43:0/1758468974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbfbc0ca0d0 con 0x7fbfbc15ceb0 2023-06-28T20:08:18.646 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.642+0000 7fbfb8ff9700 1 -- 172.21.15.43:0/1758468974 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbfbc0ca0d0 con 0x7fbfbc15ceb0 2023-06-28T20:08:18.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.647+0000 7fbfc277e700 1 -- 172.21.15.43:0/1758468974 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbf9c06ace0 msgr2=0x7fbf9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:18.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.647+0000 7fbfc277e700 1 --2- 172.21.15.43:0/1758468974 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbf9c06ace0 0x7fbf9c06d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009b40 tx=0x7fbfac009210 comp rx=0 tx=0).stop 2023-06-28T20:08:18.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.647+0000 7fbfc277e700 1 -- 172.21.15.43:0/1758468974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfbc15ceb0 msgr2=0x7fbfbc158df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:18.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.647+0000 7fbfc277e700 1 --2- 172.21.15.43:0/1758468974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfbc15ceb0 0x7fbfbc158df0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc06b260 tx=0x7fbfb4004ad0 comp rx=0 tx=0).stop 2023-06-28T20:08:18.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.647+0000 7fbfc277e700 1 -- 172.21.15.43:0/1758468974 shutdown_connections 2023-06-28T20:08:18.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.647+0000 7fbfc277e700 1 --2- 172.21.15.43:0/1758468974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbfbc15ceb0 0x7fbfbc158df0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:18.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.647+0000 7fbfc277e700 1 --2- 172.21.15.43:0/1758468974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfbc0b4f20 0x7fbfbc0cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:18.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.647+0000 7fbfc277e700 1 --2- 172.21.15.43:0/1758468974 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbf9c06ace0 0x7fbf9c06d190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:18.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.647+0000 7fbfc277e700 1 --2- 172.21.15.43:0/1758468974 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbfbc0d0e20 0x7fbfbc157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:18.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.647+0000 7fbfc277e700 1 -- 172.21.15.43:0/1758468974 >> 172.21.15.43:0/1758468974 conn(0x7fbfbc06a5c0 msgr2=0x7fbfbc0cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:18.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.647+0000 7fbfc277e700 1 -- 172.21.15.43:0/1758468974 shutdown_connections 2023-06-28T20:08:18.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.647+0000 7fbfc277e700 1 -- 172.21.15.43:0/1758468974 wait complete. 2023-06-28T20:08:18.655 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2023-06-28T20:08:18.668 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-06-28T20:08:18.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.803+0000 7f0b492fa700 1 Processor -- start 2023-06-28T20:08:18.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.803+0000 7f0b492fa700 1 -- start start 2023-06-28T20:08:18.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.804+0000 7f0b492fa700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0b440cf830 0x7f0b440cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:18.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.804+0000 7f0b492fa700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b440b89f0 con 0x7f0b4415cf10 2023-06-28T20:08:18.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.804+0000 7f0b492fa700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b440d0f00 con 0x7f0b440d0a80 2023-06-28T20:08:18.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.804+0000 7f0b492fa700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b440d0140 con 0x7f0b440cf830 2023-06-28T20:08:18.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.804+0000 7f0b42ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0b440cf830 0x7f0b440cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:18.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.804+0000 7f0b437fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0b4415cf10 0x7f0b440b82e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44090/0 (socket says 172.21.15.43:44090) 2023-06-28T20:08:18.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.804+0000 7f0b437fe700 1 -- 172.21.15.43:0/1398208466 learned_addr learned my addr 172.21.15.43:0/1398208466 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:18.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.804+0000 7f0b42ffd700 1 -- 172.21.15.43:0/1398208466 >> v1:172.21.15.178:6789/0 conn(0x7f0b440d0a80 legacy=0x7f0b440d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:18.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.804+0000 7f0b42ffd700 1 -- 172.21.15.43:0/1398208466 >> v1:172.21.15.43:6789/0 conn(0x7f0b4415cf10 legacy=0x7f0b440b82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:18.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.804+0000 7f0b42ffd700 1 -- 172.21.15.43:0/1398208466 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b440b8c80 con 0x7f0b440cf830 2023-06-28T20:08:18.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.804+0000 7f0b41ffb700 1 -- 172.21.15.43:0/1398208466 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1318471481 0 0) 0x7f0b440b89f0 con 0x7f0b4415cf10 2023-06-28T20:08:18.813 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.804+0000 7f0b42ffd700 1 --2- 172.21.15.43:0/1398208466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b440cf830 0x7f0b440cfc00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009840 tx=0x7f0b38009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d282b2ef3b5bb027 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:18.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.805+0000 7f0b41ffb700 1 -- 172.21.15.43:0/1398208466 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b38004230 con 0x7f0b440cf830 2023-06-28T20:08:18.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.805+0000 7f0b41ffb700 1 -- 172.21.15.43:0/1398208466 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b38004390 con 0x7f0b440cf830 2023-06-28T20:08:18.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.805+0000 7f0b41ffb700 1 -- 172.21.15.43:0/1398208466 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b380036a0 con 0x7f0b440cf830 2023-06-28T20:08:18.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.805+0000 7f0b492fa700 1 -- 172.21.15.43:0/1398208466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b440cf830 msgr2=0x7f0b440cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:18.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.805+0000 7f0b492fa700 1 --2- 172.21.15.43:0/1398208466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b440cf830 0x7f0b440cfc00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009840 tx=0x7f0b38009b50 comp rx=0 tx=0).stop 2023-06-28T20:08:18.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.805+0000 7f0b492fa700 1 -- 172.21.15.43:0/1398208466 shutdown_connections 2023-06-28T20:08:18.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.805+0000 7f0b492fa700 1 --2- 172.21.15.43:0/1398208466 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b440cf830 0x7f0b440cfc00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:18.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.805+0000 7f0b492fa700 1 -- 172.21.15.43:0/1398208466 >> 172.21.15.43:0/1398208466 conn(0x7f0b4406a640 msgr2=0x7f0b4406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:18.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.805+0000 7f0b492fa700 1 -- 172.21.15.43:0/1398208466 shutdown_connections 2023-06-28T20:08:18.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.805+0000 7f0b492fa700 1 -- 172.21.15.43:0/1398208466 wait complete. 2023-06-28T20:08:18.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.807+0000 7f0b492fa700 1 Processor -- start 2023-06-28T20:08:18.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.807+0000 7f0b492fa700 1 -- start start 2023-06-28T20:08:18.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.808+0000 7f0b492fa700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b440cf830 0x7f0b4415a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:18.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.808+0000 7f0b492fa700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b440d0a80 0x7f0b440cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:18.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.808+0000 7f0b492fa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b4415cf10 0x7f0b440c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:18.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.808+0000 7f0b492fa700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b440b4a40 con 0x7f0b440d0a80 2023-06-28T20:08:18.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.808+0000 7f0b492fa700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b440b4d20 con 0x7f0b4415cf10 2023-06-28T20:08:18.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.808+0000 7f0b492fa700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b440b5000 con 0x7f0b440cf830 2023-06-28T20:08:18.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.808+0000 7f0b427fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b440d0a80 0x7f0b440cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:18.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.808+0000 7f0b427fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b440d0a80 0x7f0b440cc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54908/0 (socket says 172.21.15.43:54908) 2023-06-28T20:08:18.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.808+0000 7f0b427fc700 1 -- 172.21.15.43:0/2788261876 learned_addr learned my addr 172.21.15.43:0/2788261876 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:18.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.808+0000 7f0b42ffd700 1 --2- 172.21.15.43:0/2788261876 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b440cf830 0x7f0b4415a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:18.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.808+0000 7f0b437fe700 1 --2- 172.21.15.43:0/2788261876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b4415cf10 0x7f0b440c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:18.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.808+0000 7f0b427fc700 1 -- 172.21.15.43:0/2788261876 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b440cf830 msgr2=0x7f0b4415a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:18.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.808+0000 7f0b427fc700 1 --2- 172.21.15.43:0/2788261876 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b440cf830 0x7f0b4415a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:18.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.808+0000 7f0b427fc700 1 -- 172.21.15.43:0/2788261876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b4415cf10 msgr2=0x7f0b440c6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:18.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.808+0000 7f0b427fc700 1 --2- 172.21.15.43:0/2788261876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b4415cf10 0x7f0b440c6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:18.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.808+0000 7f0b427fc700 1 -- 172.21.15.43:0/2788261876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b380094a0 con 0x7f0b440d0a80 2023-06-28T20:08:18.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.808+0000 7f0b437fe700 1 --2- 172.21.15.43:0/2788261876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b4415cf10 0x7f0b440c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:18.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.808+0000 7f0b42ffd700 1 --2- 172.21.15.43:0/2788261876 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b440cf830 0x7f0b4415a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:18.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.809+0000 7f0b427fc700 1 --2- 172.21.15.43:0/2788261876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b440d0a80 0x7f0b440cc710 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400b450 tx=0x7f0b3400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:18.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.809+0000 7f0b2b7fe700 1 -- 172.21.15.43:0/2788261876 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34011830 con 0x7f0b440d0a80 2023-06-28T20:08:18.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.809+0000 7f0b2b7fe700 1 -- 172.21.15.43:0/2788261876 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b34011990 con 0x7f0b440d0a80 2023-06-28T20:08:18.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.809+0000 7f0b492fa700 1 -- 172.21.15.43:0/2788261876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b440b52e0 con 0x7f0b440d0a80 2023-06-28T20:08:18.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.809+0000 7f0b2b7fe700 1 -- 172.21.15.43:0/2788261876 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b34011d40 con 0x7f0b440d0a80 2023-06-28T20:08:18.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.809+0000 7f0b492fa700 1 -- 172.21.15.43:0/2788261876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b440ccc50 con 0x7f0b440d0a80 2023-06-28T20:08:18.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.810+0000 7f0b492fa700 1 -- 172.21.15.43:0/2788261876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b440c9eb0 con 0x7f0b440d0a80 2023-06-28T20:08:18.822 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.810+0000 7f0b2b7fe700 1 -- 172.21.15.43:0/2788261876 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0b3401a400 con 0x7f0b440d0a80 2023-06-28T20:08:18.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.811+0000 7f0b2b7fe700 1 --2- 172.21.15.43:0/2788261876 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0b3006ace0 0x7f0b3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:18.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.811+0000 7f0b2b7fe700 1 -- 172.21.15.43:0/2788261876 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f0b34089000 con 0x7f0b440d0a80 2023-06-28T20:08:18.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.811+0000 7f0b42ffd700 1 --2- 172.21.15.43:0/2788261876 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0b3006ace0 0x7f0b3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:18.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.811+0000 7f0b42ffd700 1 --2- 172.21.15.43:0/2788261876 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0b3006ace0 0x7f0b3006d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0b380090b0 tx=0x7f0b38013040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:18.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:18.816+0000 7f0b2b7fe700 1 -- 172.21.15.43:0/2788261876 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0b340542a0 con 0x7f0b440d0a80 2023-06-28T20:08:19.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.014+0000 7f0b492fa700 1 -- 172.21.15.43:0/2788261876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0b440728b0 con 0x7f0b440d0a80 2023-06-28T20:08:19.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.651+0000 7f0b2b7fe700 1 -- 172.21.15.43:0/2788261876 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0b340578c0 con 0x7f0b440d0a80 2023-06-28T20:08:19.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.661+0000 7f0b492fa700 1 -- 172.21.15.43:0/2788261876 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0b3006ace0 msgr2=0x7f0b3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:19.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.661+0000 7f0b492fa700 1 --2- 172.21.15.43:0/2788261876 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0b3006ace0 0x7f0b3006d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0b380090b0 tx=0x7f0b38013040 comp rx=0 tx=0).stop 2023-06-28T20:08:19.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.661+0000 7f0b492fa700 1 -- 172.21.15.43:0/2788261876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b440d0a80 msgr2=0x7f0b440cc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:19.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.661+0000 7f0b492fa700 1 --2- 172.21.15.43:0/2788261876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b440d0a80 0x7f0b440cc710 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400b450 tx=0x7f0b3400b810 comp rx=0 tx=0).stop 2023-06-28T20:08:19.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.662+0000 7f0b492fa700 1 -- 172.21.15.43:0/2788261876 shutdown_connections 2023-06-28T20:08:19.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.662+0000 7f0b492fa700 1 --2- 172.21.15.43:0/2788261876 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b4415cf10 0x7f0b440c6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:19.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.662+0000 7f0b492fa700 1 --2- 172.21.15.43:0/2788261876 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0b3006ace0 0x7f0b3006d190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:19.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.662+0000 7f0b492fa700 1 --2- 172.21.15.43:0/2788261876 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b440cf830 0x7f0b4415a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:19.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.662+0000 7f0b492fa700 1 --2- 172.21.15.43:0/2788261876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b440d0a80 0x7f0b440cc710 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:19.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.662+0000 7f0b492fa700 1 -- 172.21.15.43:0/2788261876 >> 172.21.15.43:0/2788261876 conn(0x7f0b4406a640 msgr2=0x7f0b440b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:19.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.663+0000 7f0b492fa700 1 -- 172.21.15.43:0/2788261876 shutdown_connections 2023-06-28T20:08:19.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.663+0000 7f0b492fa700 1 -- 172.21.15.43:0/2788261876 wait complete. 2023-06-28T20:08:19.672 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2023-06-28T20:08:19.684 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-06-28T20:08:19.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.814+0000 7fd285a00700 1 Processor -- start 2023-06-28T20:08:19.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.814+0000 7fd285a00700 1 -- start start 2023-06-28T20:08:19.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.815+0000 7fd285a00700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd2800b1530 0x7fd2800b1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:19.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.815+0000 7fd285a00700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2801586f0 0x7fd280158b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:19.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.815+0000 7fd285a00700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd28006c9c0 con 0x7fd2800aed20 2023-06-28T20:08:19.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.815+0000 7fd285a00700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2800b1ec0 con 0x7fd2800b1530 2023-06-28T20:08:19.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.815+0000 7fd285a00700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28015d920 con 0x7fd2801586f0 2023-06-28T20:08:19.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.815+0000 7fd27effd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd2800aed20 0x7fd2800caa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44100/0 (socket says 172.21.15.43:44100) 2023-06-28T20:08:19.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.815+0000 7fd27e7fc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd2800b1530 0x7fd2800b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:19.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.815+0000 7fd27effd700 1 -- 172.21.15.43:0/910446240 learned_addr learned my addr 172.21.15.43:0/910446240 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:19.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.815+0000 7fd27f7fe700 1 --2- 172.21.15.43:0/910446240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2801586f0 0x7fd280158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:19.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.816+0000 7fd27dffb700 1 -- 172.21.15.43:0/910446240 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2764366419 0 0) 0x7fd28006c9c0 con 0x7fd2800aed20 2023-06-28T20:08:19.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.816+0000 7fd27dffb700 1 -- 172.21.15.43:0/910446240 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2680035b0 con 0x7fd2800aed20 2023-06-28T20:08:19.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.816+0000 7fd27e7fc700 1 -- 172.21.15.43:0/910446240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2801586f0 msgr2=0x7fd280158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:19.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.816+0000 7fd27e7fc700 1 --2- 172.21.15.43:0/910446240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2801586f0 0x7fd280158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:19.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.816+0000 7fd27e7fc700 1 -- 172.21.15.43:0/910446240 >> v1:172.21.15.43:6789/0 conn(0x7fd2800aed20 legacy=0x7fd2800caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:19.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.816+0000 7fd27e7fc700 1 -- 172.21.15.43:0/910446240 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28015cd10 con 0x7fd2800b1530 2023-06-28T20:08:19.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.816+0000 7fd27e7fc700 1 --2- 172.21.15.43:0/910446240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2800b1530 0x7fd2800b1980 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd270009840 tx=0x7fd270009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d819bd0ba3411de server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:19.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.816+0000 7fd27dffb700 1 -- 172.21.15.43:0/910446240 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd270011830 con 0x7fd2800b1530 2023-06-28T20:08:19.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.816+0000 7fd27dffb700 1 -- 172.21.15.43:0/910446240 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd270011990 con 0x7fd2800b1530 2023-06-28T20:08:19.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.816+0000 7fd27dffb700 1 -- 172.21.15.43:0/910446240 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd270011d20 con 0x7fd2800b1530 2023-06-28T20:08:19.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.817+0000 7fd285a00700 1 -- 172.21.15.43:0/910446240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2800b1530 msgr2=0x7fd2800b1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:19.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.817+0000 7fd285a00700 1 --2- 172.21.15.43:0/910446240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2800b1530 0x7fd2800b1980 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd270009840 tx=0x7fd270009b50 comp rx=0 tx=0).stop 2023-06-28T20:08:19.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.817+0000 7fd285a00700 1 -- 172.21.15.43:0/910446240 shutdown_connections 2023-06-28T20:08:19.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.817+0000 7fd285a00700 1 --2- 172.21.15.43:0/910446240 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2801586f0 0x7fd280158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:19.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.817+0000 7fd285a00700 1 --2- 172.21.15.43:0/910446240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2800b1530 0x7fd2800b1980 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:19.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.817+0000 7fd285a00700 1 -- 172.21.15.43:0/910446240 >> 172.21.15.43:0/910446240 conn(0x7fd2800cfab0 msgr2=0x7fd2800cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:19.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.817+0000 7fd285a00700 1 -- 172.21.15.43:0/910446240 shutdown_connections 2023-06-28T20:08:19.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.817+0000 7fd285a00700 1 -- 172.21.15.43:0/910446240 wait complete. 2023-06-28T20:08:19.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.818+0000 7fd285a00700 1 Processor -- start 2023-06-28T20:08:19.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.818+0000 7fd285a00700 1 -- start start 2023-06-28T20:08:19.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.819+0000 7fd285a00700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2800aed20 0x7fd2800c3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:19.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.819+0000 7fd285a00700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2800b1530 0x7fd2800c3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:19.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.819+0000 7fd285a00700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2801586f0 0x7fd2801556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:19.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.819+0000 7fd285a00700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2801f1ba0 con 0x7fd2801586f0 2023-06-28T20:08:19.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.819+0000 7fd285a00700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2801f5ec0 con 0x7fd2800b1530 2023-06-28T20:08:19.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.819+0000 7fd285a00700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2801f1d90 con 0x7fd2800aed20 2023-06-28T20:08:19.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.819+0000 7fd27f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2801586f0 0x7fd2801556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:19.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.819+0000 7fd27f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2801586f0 0x7fd2801556c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54924/0 (socket says 172.21.15.43:54924) 2023-06-28T20:08:19.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.819+0000 7fd27f7fe700 1 -- 172.21.15.43:0/67779572 learned_addr learned my addr 172.21.15.43:0/67779572 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:19.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.819+0000 7fd27effd700 1 --2- 172.21.15.43:0/67779572 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2800aed20 0x7fd2800c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:19.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.819+0000 7fd27e7fc700 1 --2- 172.21.15.43:0/67779572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2800b1530 0x7fd2800c3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:19.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.819+0000 7fd27f7fe700 1 -- 172.21.15.43:0/67779572 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2800aed20 msgr2=0x7fd2800c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:19.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.819+0000 7fd27f7fe700 1 --2- 172.21.15.43:0/67779572 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2800aed20 0x7fd2800c3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:19.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.819+0000 7fd27f7fe700 1 -- 172.21.15.43:0/67779572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2800b1530 msgr2=0x7fd2800c3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:19.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.819+0000 7fd27f7fe700 1 --2- 172.21.15.43:0/67779572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2800b1530 0x7fd2800c3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:19.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.819+0000 7fd27f7fe700 1 -- 172.21.15.43:0/67779572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2700094a0 con 0x7fd2801586f0 2023-06-28T20:08:19.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.820+0000 7fd27f7fe700 1 --2- 172.21.15.43:0/67779572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2801586f0 0x7fd2801556c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd278007e60 tx=0x7fd27800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:19.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.820+0000 7fd267fff700 1 -- 172.21.15.43:0/67779572 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd27800ede0 con 0x7fd2801586f0 2023-06-28T20:08:19.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.820+0000 7fd267fff700 1 -- 172.21.15.43:0/67779572 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd278014420 con 0x7fd2801586f0 2023-06-28T20:08:19.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.820+0000 7fd285a00700 1 -- 172.21.15.43:0/67779572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2801f2070 con 0x7fd2801586f0 2023-06-28T20:08:19.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.820+0000 7fd267fff700 1 -- 172.21.15.43:0/67779572 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2780147d0 con 0x7fd2801586f0 2023-06-28T20:08:19.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.820+0000 7fd285a00700 1 -- 172.21.15.43:0/67779572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2801f2870 con 0x7fd2801586f0 2023-06-28T20:08:19.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.821+0000 7fd285a00700 1 -- 172.21.15.43:0/67779572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd280067a50 con 0x7fd2801586f0 2023-06-28T20:08:19.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.821+0000 7fd267fff700 1 -- 172.21.15.43:0/67779572 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd278014930 con 0x7fd2801586f0 2023-06-28T20:08:19.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.822+0000 7fd267fff700 1 --2- 172.21.15.43:0/67779572 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd26c06ace0 0x7fd26c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:19.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.822+0000 7fd267fff700 1 -- 172.21.15.43:0/67779572 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fd278089c00 con 0x7fd2801586f0 2023-06-28T20:08:19.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.822+0000 7fd27effd700 1 --2- 172.21.15.43:0/67779572 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd26c06ace0 0x7fd26c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:19.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.822+0000 7fd27effd700 1 --2- 172.21.15.43:0/67779572 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd26c06ace0 0x7fd26c06d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd2740011e0 tx=0x7fd274009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:19.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:19.826+0000 7fd267fff700 1 -- 172.21.15.43:0/67779572 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd278090080 con 0x7fd2801586f0 2023-06-28T20:08:20.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.071+0000 7fd285a00700 1 -- 172.21.15.43:0/67779572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd2801f2590 con 0x7fd2801586f0 2023-06-28T20:08:20.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.657+0000 7fd267fff700 1 -- 172.21.15.43:0/67779572 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd278054dc0 con 0x7fd2801586f0 2023-06-28T20:08:20.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.662+0000 7fd285a00700 1 -- 172.21.15.43:0/67779572 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd26c06ace0 msgr2=0x7fd26c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:20.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.662+0000 7fd285a00700 1 --2- 172.21.15.43:0/67779572 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd26c06ace0 0x7fd26c06d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd2740011e0 tx=0x7fd274009040 comp rx=0 tx=0).stop 2023-06-28T20:08:20.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.662+0000 7fd285a00700 1 -- 172.21.15.43:0/67779572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2801586f0 msgr2=0x7fd2801556c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:20.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.662+0000 7fd285a00700 1 --2- 172.21.15.43:0/67779572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2801586f0 0x7fd2801556c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fd278007e60 tx=0x7fd27800e3c0 comp rx=0 tx=0).stop 2023-06-28T20:08:20.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.662+0000 7fd285a00700 1 -- 172.21.15.43:0/67779572 shutdown_connections 2023-06-28T20:08:20.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.662+0000 7fd285a00700 1 --2- 172.21.15.43:0/67779572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2801586f0 0x7fd2801556c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:20.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.662+0000 7fd285a00700 1 --2- 172.21.15.43:0/67779572 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd26c06ace0 0x7fd26c06d190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:20.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.662+0000 7fd285a00700 1 --2- 172.21.15.43:0/67779572 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd2800aed20 0x7fd2800c3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:20.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.662+0000 7fd285a00700 1 --2- 172.21.15.43:0/67779572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2800b1530 0x7fd2800c3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:20.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.662+0000 7fd285a00700 1 -- 172.21.15.43:0/67779572 >> 172.21.15.43:0/67779572 conn(0x7fd2800cfab0 msgr2=0x7fd280158b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:20.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.663+0000 7fd285a00700 1 -- 172.21.15.43:0/67779572 shutdown_connections 2023-06-28T20:08:20.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.663+0000 7fd285a00700 1 -- 172.21.15.43:0/67779572 wait complete. 2023-06-28T20:08:20.672 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 5 and data pool 6 2023-06-28T20:08:20.695 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:08:20.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.820+0000 7f3ffe351700 1 Processor -- start 2023-06-28T20:08:20.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.820+0000 7f3ffe351700 1 -- start start 2023-06-28T20:08:20.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.821+0000 7f3ffe351700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ff815d9f0 0x7f3ff806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:20.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.821+0000 7f3ffe351700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3ff8070cb0 0x7f3ff8069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:20.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.821+0000 7f3ffe351700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff8071170 con 0x7f3ff806a430 2023-06-28T20:08:20.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.821+0000 7f3ffe351700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff806a8d0 con 0x7f3ff8070cb0 2023-06-28T20:08:20.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.821+0000 7f3ffe351700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff815ddc0 con 0x7f3ff815d9f0 2023-06-28T20:08:20.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.821+0000 7f3ff7fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ff815d9f0 0x7f3ff806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:20.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.821+0000 7f3ff77fe700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3ff8070cb0 0x7f3ff8069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:20.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.821+0000 7f3ffc8ee700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f3ff806a430 0x7f3ff815df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:43258/0 (socket says 172.21.15.43:43258) 2023-06-28T20:08:20.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.821+0000 7f3ff7fff700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ff815d9f0 0x7f3ff806d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.43:38916/0 (socket says 172.21.15.43:38916) 2023-06-28T20:08:20.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.821+0000 7f3ffc8ee700 1 -- 172.21.15.43:0/3157485188 learned_addr learned my addr 172.21.15.43:0/3157485188 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:20.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.822+0000 7f3ff6ffd700 1 -- 172.21.15.43:0/3157485188 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812840172 0 0) 0x7f3ff8071170 con 0x7f3ff806a430 2023-06-28T20:08:20.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.822+0000 7f3ff6ffd700 1 -- 172.21.15.43:0/3157485188 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fe40035b0 con 0x7f3ff806a430 2023-06-28T20:08:20.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.822+0000 7f3ff7fff700 1 -- 172.21.15.43:0/3157485188 >> v2:172.21.15.178:3300/0 conn(0x7f3ff8070cb0 msgr2=0x7f3ff8069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:20.829 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.822+0000 7f3ff7fff700 1 --2- 172.21.15.43:0/3157485188 >> v2:172.21.15.178:3300/0 conn(0x7f3ff8070cb0 0x7f3ff8069ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:20.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.822+0000 7f3ff7fff700 1 -- 172.21.15.43:0/3157485188 >> v1:172.21.15.178:6789/0 conn(0x7f3ff806a430 legacy=0x7f3ff815df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:20.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.822+0000 7f3ff7fff700 1 -- 172.21.15.43:0/3157485188 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff8071450 con 0x7f3ff815d9f0 2023-06-28T20:08:20.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.822+0000 7f3ff77fe700 1 --2- 172.21.15.43:0/3157485188 >> v2:172.21.15.178:3300/0 conn(0x7f3ff8070cb0 0x7f3ff8069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:08:20.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.822+0000 7f3ff7fff700 1 --2- 172.21.15.43:0/3157485188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ff815d9f0 0x7f3ff806d3b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009840 tx=0x7f3fe8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7452df4508369882 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:20.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.822+0000 7f3ff6ffd700 1 -- 172.21.15.43:0/3157485188 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8011830 con 0x7f3ff815d9f0 2023-06-28T20:08:20.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.822+0000 7f3ff6ffd700 1 -- 172.21.15.43:0/3157485188 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe8011990 con 0x7f3ff815d9f0 2023-06-28T20:08:20.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.822+0000 7f3ff6ffd700 1 -- 172.21.15.43:0/3157485188 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8011d00 con 0x7f3ff815d9f0 2023-06-28T20:08:20.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.823+0000 7f3ffe351700 1 -- 172.21.15.43:0/3157485188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ff815d9f0 msgr2=0x7f3ff806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:20.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.823+0000 7f3ffe351700 1 --2- 172.21.15.43:0/3157485188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ff815d9f0 0x7f3ff806d3b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009840 tx=0x7f3fe8009b50 comp rx=0 tx=0).stop 2023-06-28T20:08:20.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.823+0000 7f3ffe351700 1 -- 172.21.15.43:0/3157485188 shutdown_connections 2023-06-28T20:08:20.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.823+0000 7f3ffe351700 1 --2- 172.21.15.43:0/3157485188 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ff815d9f0 0x7f3ff806d3b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:20.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.823+0000 7f3ffe351700 1 --2- 172.21.15.43:0/3157485188 >> v2:172.21.15.178:3300/0 conn(0x7f3ff8070cb0 0x7f3ff8069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:20.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.823+0000 7f3ffe351700 1 -- 172.21.15.43:0/3157485188 >> 172.21.15.43:0/3157485188 conn(0x7f3ff80b1720 msgr2=0x7f3ff80b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:20.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.823+0000 7f3ffe351700 1 -- 172.21.15.43:0/3157485188 shutdown_connections 2023-06-28T20:08:20.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.823+0000 7f3ffe351700 1 -- 172.21.15.43:0/3157485188 wait complete. 2023-06-28T20:08:20.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.824+0000 7f3ffe351700 1 Processor -- start 2023-06-28T20:08:20.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.824+0000 7f3ffe351700 1 -- start start 2023-06-28T20:08:20.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.825+0000 7f3ffe351700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ff806a430 0x7f3ff8155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:20.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.825+0000 7f3ffe351700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff8070cb0 0x7f3ff80c48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:20.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.825+0000 7f3ffe351700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ff815d9f0 0x7f3ff815a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:20.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.825+0000 7f3ffe351700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff815a510 con 0x7f3ff8070cb0 2023-06-28T20:08:20.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.825+0000 7f3ffe351700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff8159890 con 0x7f3ff806a430 2023-06-28T20:08:20.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.825+0000 7f3ffe351700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff8159b70 con 0x7f3ff815d9f0 2023-06-28T20:08:20.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.825+0000 7f3ff77fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff8070cb0 0x7f3ff80c48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:20.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.825+0000 7f3ff77fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff8070cb0 0x7f3ff80c48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54952/0 (socket says 172.21.15.43:54952) 2023-06-28T20:08:20.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.825+0000 7f3ff77fe700 1 -- 172.21.15.43:0/2257505909 learned_addr learned my addr 172.21.15.43:0/2257505909 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:20.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.825+0000 7f3ffc8ee700 1 --2- 172.21.15.43:0/2257505909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ff815d9f0 0x7f3ff815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:20.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.825+0000 7f3ff7fff700 1 --2- 172.21.15.43:0/2257505909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ff806a430 0x7f3ff8155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:20.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.825+0000 7f3ff77fe700 1 -- 172.21.15.43:0/2257505909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ff815d9f0 msgr2=0x7f3ff815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:20.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.825+0000 7f3ff77fe700 1 --2- 172.21.15.43:0/2257505909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ff815d9f0 0x7f3ff815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:20.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.825+0000 7f3ff77fe700 1 -- 172.21.15.43:0/2257505909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ff806a430 msgr2=0x7f3ff8155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:20.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.825+0000 7f3ff77fe700 1 --2- 172.21.15.43:0/2257505909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ff806a430 0x7f3ff8155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:20.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.825+0000 7f3ff77fe700 1 -- 172.21.15.43:0/2257505909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe00094a0 con 0x7f3ff8070cb0 2023-06-28T20:08:20.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.826+0000 7f3ff77fe700 1 --2- 172.21.15.43:0/2257505909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff8070cb0 0x7f3ff80c48e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000ebe0 tx=0x7f3fe000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:20.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.826+0000 7f3ff4ff9700 1 -- 172.21.15.43:0/2257505909 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe00097a0 con 0x7f3ff8070cb0 2023-06-28T20:08:20.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.826+0000 7f3ff4ff9700 1 -- 172.21.15.43:0/2257505909 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe000c850 con 0x7f3ff8070cb0 2023-06-28T20:08:20.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.826+0000 7f3ffe351700 1 -- 172.21.15.43:0/2257505909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe80094a0 con 0x7f3ff8070cb0 2023-06-28T20:08:20.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.826+0000 7f3ff4ff9700 1 -- 172.21.15.43:0/2257505909 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe000cbe0 con 0x7f3ff8070cb0 2023-06-28T20:08:20.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.826+0000 7f3ffe351700 1 -- 172.21.15.43:0/2257505909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff815a130 con 0x7f3ff8070cb0 2023-06-28T20:08:20.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.827+0000 7f3ffe351700 1 -- 172.21.15.43:0/2257505909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff80728b0 con 0x7f3ff8070cb0 2023-06-28T20:08:20.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.827+0000 7f3ff4ff9700 1 -- 172.21.15.43:0/2257505909 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3fe000cd40 con 0x7f3ff8070cb0 2023-06-28T20:08:20.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.828+0000 7f3ff4ff9700 1 --2- 172.21.15.43:0/2257505909 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f3fd806ace0 0x7f3fd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:20.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.828+0000 7f3ff4ff9700 1 -- 172.21.15.43:0/2257505909 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3fe00891b0 con 0x7f3ff8070cb0 2023-06-28T20:08:20.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.828+0000 7f3ff7fff700 1 --2- 172.21.15.43:0/2257505909 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f3fd806ace0 0x7f3fd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:20.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.828+0000 7f3ff7fff700 1 --2- 172.21.15.43:0/2257505909 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f3fd806ace0 0x7f3fd806d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8000c00 tx=0x7f3fe801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:20.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:20.832+0000 7f3ff4ff9700 1 -- 172.21.15.43:0/2257505909 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3fe0054250 con 0x7f3ff8070cb0 2023-06-28T20:08:21.036 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.032+0000 7f3ffe351700 1 -- 172.21.15.43:0/2257505909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ff806f080 con 0x7f3ff8070cb0 2023-06-28T20:08:21.038 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.034+0000 7f3ff4ff9700 1 -- 172.21.15.43:0/2257505909 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+18306 (secure 0 0 0) 0x7f3fe0057870 con 0x7f3ff8070cb0 2023-06-28T20:08:21.038 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:21.038 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":34,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:08:20.073237+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:08:17.811454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-06-28T20:08:19.046351+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:08:21.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.036+0000 7f3ffe351700 1 -- 172.21.15.43:0/2257505909 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f3fd806ace0 msgr2=0x7f3fd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:21.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.036+0000 7f3ffe351700 1 --2- 172.21.15.43:0/2257505909 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f3fd806ace0 0x7f3fd806d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8000c00 tx=0x7f3fe801c040 comp rx=0 tx=0).stop 2023-06-28T20:08:21.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.036+0000 7f3ffe351700 1 -- 172.21.15.43:0/2257505909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff8070cb0 msgr2=0x7f3ff80c48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:21.040 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.036+0000 7f3ffe351700 1 --2- 172.21.15.43:0/2257505909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff8070cb0 0x7f3ff80c48e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000ebe0 tx=0x7f3fe000eef0 comp rx=0 tx=0).stop 2023-06-28T20:08:21.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.036+0000 7f3ffe351700 1 -- 172.21.15.43:0/2257505909 shutdown_connections 2023-06-28T20:08:21.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.036+0000 7f3ffe351700 1 --2- 172.21.15.43:0/2257505909 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f3fd806ace0 0x7f3fd806d190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:21.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.036+0000 7f3ffe351700 1 --2- 172.21.15.43:0/2257505909 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3ff815d9f0 0x7f3ff815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:21.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.036+0000 7f3ffe351700 1 --2- 172.21.15.43:0/2257505909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ff806a430 0x7f3ff8155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:21.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.036+0000 7f3ffe351700 1 --2- 172.21.15.43:0/2257505909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff8070cb0 0x7f3ff80c48e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:21.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.036+0000 7f3ffe351700 1 -- 172.21.15.43:0/2257505909 >> 172.21.15.43:0/2257505909 conn(0x7f3ff80b1720 msgr2=0x7f3ff80b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:21.041 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.036+0000 7f3ffe351700 1 -- 172.21.15.43:0/2257505909 shutdown_connections 2023-06-28T20:08:21.042 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.036+0000 7f3ffe351700 1 -- 172.21.15.43:0/2257505909 wait complete. 2023-06-28T20:08:21.053 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:08:21.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.174+0000 7f7f0a394700 1 Processor -- start 2023-06-28T20:08:21.178 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.174+0000 7f7f0a394700 1 -- start start 2023-06-28T20:08:21.179 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.175+0000 7f7f0a394700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f0406ca80 0x7f7f040cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:21.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.175+0000 7f7f0a394700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7f0406a600 0x7f7f040ac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:21.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.175+0000 7f7f0a394700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f04071410 con 0x7f7f040af060 2023-06-28T20:08:21.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.175+0000 7f7f0a394700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f040716b0 con 0x7f7f0406a600 2023-06-28T20:08:21.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.175+0000 7f7f0a394700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f04071990 con 0x7f7f0406ca80 2023-06-28T20:08:21.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.175+0000 7f7f08931700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7f0406a600 0x7f7f040ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:21.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.175+0000 7f7f08931700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7f0406a600 0x7f7f040ac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54962/0 (socket says 172.21.15.43:54962) 2023-06-28T20:08:21.180 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.175+0000 7f7f08931700 1 -- 172.21.15.43:0/4108029051 learned_addr learned my addr 172.21.15.43:0/4108029051 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.175+0000 7f7f037fe700 1 --2- 172.21.15.43:0/4108029051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f0406ca80 0x7f7f040cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.175+0000 7f7f08931700 1 -- 172.21.15.43:0/4108029051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f0406ca80 msgr2=0x7f7f040cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.175+0000 7f7f08931700 1 --2- 172.21.15.43:0/4108029051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f0406ca80 0x7f7f040cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.175+0000 7f7f08931700 1 -- 172.21.15.43:0/4108029051 >> v1:172.21.15.178:6789/0 conn(0x7f7f040af060 legacy=0x7f7f04069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.175+0000 7f7f08931700 1 -- 172.21.15.43:0/4108029051 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f040af430 con 0x7f7f0406a600 2023-06-28T20:08:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.175+0000 7f7f08931700 1 --2- 172.21.15.43:0/4108029051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f0406a600 0x7f7f040ac870 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009e50 tx=0x7f7efc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942c3e83f8ca01b0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.175+0000 7f7f02ffd700 1 -- 172.21.15.43:0/4108029051 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc00e070 con 0x7f7f0406a600 2023-06-28T20:08:21.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.176+0000 7f7f02ffd700 1 -- 172.21.15.43:0/4108029051 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7efc00b9c0 con 0x7f7f0406a600 2023-06-28T20:08:21.182 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.176+0000 7f7f02ffd700 1 -- 172.21.15.43:0/4108029051 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc00bd70 con 0x7f7f0406a600 2023-06-28T20:08:21.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.176+0000 7f7f0a394700 1 -- 172.21.15.43:0/4108029051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f0406a600 msgr2=0x7f7f040ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:21.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.176+0000 7f7f0a394700 1 --2- 172.21.15.43:0/4108029051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f0406a600 0x7f7f040ac870 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009e50 tx=0x7f7efc00b330 comp rx=0 tx=0).stop 2023-06-28T20:08:21.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.176+0000 7f7f0a394700 1 -- 172.21.15.43:0/4108029051 shutdown_connections 2023-06-28T20:08:21.184 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.176+0000 7f7f0a394700 1 --2- 172.21.15.43:0/4108029051 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f0406a600 0x7f7f040ac870 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:21.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.176+0000 7f7f0a394700 1 --2- 172.21.15.43:0/4108029051 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f0406ca80 0x7f7f040cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:21.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.176+0000 7f7f0a394700 1 -- 172.21.15.43:0/4108029051 >> 172.21.15.43:0/4108029051 conn(0x7f7f040b1720 msgr2=0x7f7f040b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:21.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.176+0000 7f7f0a394700 1 -- 172.21.15.43:0/4108029051 shutdown_connections 2023-06-28T20:08:21.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.176+0000 7f7f0a394700 1 -- 172.21.15.43:0/4108029051 wait complete. 2023-06-28T20:08:21.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.177+0000 7f7f0a394700 1 Processor -- start 2023-06-28T20:08:21.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.178+0000 7f7f0a394700 1 -- start start 2023-06-28T20:08:21.185 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.178+0000 7f7f0a394700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f0406a600 0x7f7f040b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:21.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.178+0000 7f7f0a394700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f040af060 0x7f7f04149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:21.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.178+0000 7f7f0a394700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f040b2a40 0x7f7f04158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:21.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.178+0000 7f7f0a394700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f04149580 con 0x7f7f040af060 2023-06-28T20:08:21.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.178+0000 7f7f0a394700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f040b2f20 con 0x7f7f0406a600 2023-06-28T20:08:21.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.178+0000 7f7f0a394700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f040b3200 con 0x7f7f040b2a40 2023-06-28T20:08:21.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.178+0000 7f7f037fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f040af060 0x7f7f04149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:21.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.178+0000 7f7f037fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f040af060 0x7f7f04149040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54964/0 (socket says 172.21.15.43:54964) 2023-06-28T20:08:21.186 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.178+0000 7f7f037fe700 1 -- 172.21.15.43:0/3690872983 learned_addr learned my addr 172.21.15.43:0/3690872983 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:21.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.178+0000 7f7f03fff700 1 --2- 172.21.15.43:0/3690872983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f0406a600 0x7f7f040b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:21.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.178+0000 7f7f08931700 1 --2- 172.21.15.43:0/3690872983 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f040b2a40 0x7f7f04158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:21.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.179+0000 7f7f037fe700 1 -- 172.21.15.43:0/3690872983 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f040b2a40 msgr2=0x7f7f04158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:21.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.179+0000 7f7f037fe700 1 --2- 172.21.15.43:0/3690872983 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f040b2a40 0x7f7f04158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:21.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.179+0000 7f7f037fe700 1 -- 172.21.15.43:0/3690872983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f0406a600 msgr2=0x7f7f040b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:21.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.179+0000 7f7f037fe700 1 --2- 172.21.15.43:0/3690872983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f0406a600 0x7f7f040b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:21.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.179+0000 7f7f037fe700 1 -- 172.21.15.43:0/3690872983 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc009b30 con 0x7f7f040af060 2023-06-28T20:08:21.187 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.179+0000 7f7f037fe700 1 --2- 172.21.15.43:0/3690872983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f040af060 0x7f7f04149040 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400e810 tx=0x7f7ef400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:21.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.179+0000 7f7f00ff9700 1 -- 172.21.15.43:0/3690872983 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef400c070 con 0x7f7f040af060 2023-06-28T20:08:21.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.179+0000 7f7f00ff9700 1 -- 172.21.15.43:0/3690872983 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef4013610 con 0x7f7f040af060 2023-06-28T20:08:21.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.179+0000 7f7f0a394700 1 -- 172.21.15.43:0/3690872983 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f040b34e0 con 0x7f7f040af060 2023-06-28T20:08:21.188 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.179+0000 7f7f00ff9700 1 -- 172.21.15.43:0/3690872983 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef40139c0 con 0x7f7f040af060 2023-06-28T20:08:21.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.179+0000 7f7f0a394700 1 -- 172.21.15.43:0/3690872983 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f040b3aa0 con 0x7f7f040af060 2023-06-28T20:08:21.189 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.180+0000 7f7f0a394700 1 -- 172.21.15.43:0/3690872983 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f040cdbc0 con 0x7f7f040af060 2023-06-28T20:08:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.181+0000 7f7f00ff9700 1 -- 172.21.15.43:0/3690872983 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7ef4013b20 con 0x7f7f040af060 2023-06-28T20:08:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.181+0000 7f7f00ff9700 1 --2- 172.21.15.43:0/3690872983 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7ee406ace0 0x7f7ee406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.181+0000 7f7f00ff9700 1 -- 172.21.15.43:0/3690872983 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7ef40891b0 con 0x7f7f040af060 2023-06-28T20:08:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.181+0000 7f7f03fff700 1 --2- 172.21.15.43:0/3690872983 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7ee406ace0 0x7f7ee406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.182+0000 7f7f03fff700 1 --2- 172.21.15.43:0/3690872983 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7ee406ace0 0x7f7ee406d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7eec007fd0 tx=0x7f7eec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.186+0000 7f7f00ff9700 1 -- 172.21.15.43:0/3690872983 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7ef4057d00 con 0x7f7f040af060 2023-06-28T20:08:21.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.384+0000 7f7f0a394700 1 -- 172.21.15.43:0/3690872983 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f040728b0 con 0x7f7f040af060 2023-06-28T20:08:21.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.385+0000 7f7f00ff9700 1 -- 172.21.15.43:0/3690872983 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+18306 (secure 0 0 0) 0x7f7ef4057890 con 0x7f7f040af060 2023-06-28T20:08:21.389 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:21.389 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":34,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:08:20.073237+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:08:17.811454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-06-28T20:08:19.046351+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:08:21.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.387+0000 7f7f0a394700 1 -- 172.21.15.43:0/3690872983 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7ee406ace0 msgr2=0x7f7ee406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:21.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.387+0000 7f7f0a394700 1 --2- 172.21.15.43:0/3690872983 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7ee406ace0 0x7f7ee406d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7eec007fd0 tx=0x7f7eec009040 comp rx=0 tx=0).stop 2023-06-28T20:08:21.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.387+0000 7f7f0a394700 1 -- 172.21.15.43:0/3690872983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f040af060 msgr2=0x7f7f04149040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:21.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.387+0000 7f7f0a394700 1 --2- 172.21.15.43:0/3690872983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f040af060 0x7f7f04149040 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7ef400e810 tx=0x7f7ef400eb20 comp rx=0 tx=0).stop 2023-06-28T20:08:21.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.387+0000 7f7f0a394700 1 -- 172.21.15.43:0/3690872983 shutdown_connections 2023-06-28T20:08:21.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.387+0000 7f7f0a394700 1 --2- 172.21.15.43:0/3690872983 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7ee406ace0 0x7f7ee406d190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:21.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.387+0000 7f7f0a394700 1 --2- 172.21.15.43:0/3690872983 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7f040b2a40 0x7f7f04158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:21.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.387+0000 7f7f0a394700 1 --2- 172.21.15.43:0/3690872983 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f0406a600 0x7f7f040b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:21.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.387+0000 7f7f0a394700 1 --2- 172.21.15.43:0/3690872983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f040af060 0x7f7f04149040 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:21.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.387+0000 7f7f0a394700 1 -- 172.21.15.43:0/3690872983 >> 172.21.15.43:0/3690872983 conn(0x7f7f040b1720 msgr2=0x7f7f0415bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:21.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.387+0000 7f7f0a394700 1 -- 172.21.15.43:0/3690872983 shutdown_connections 2023-06-28T20:08:21.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.387+0000 7f7f0a394700 1 -- 172.21.15.43:0/3690872983 wait complete. 2023-06-28T20:08:21.405 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-06-28T20:08:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.529+0000 7f897746a700 1 Processor -- start 2023-06-28T20:08:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.529+0000 7f897746a700 1 -- start start 2023-06-28T20:08:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.530+0000 7f897746a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89700b1f90 0x7f8970160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.530+0000 7f897746a700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8970160250 con 0x7f89700af060 2023-06-28T20:08:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.530+0000 7f897746a700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8970071410 con 0x7f89700aec90 2023-06-28T20:08:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.530+0000 7f897746a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897015cb30 con 0x7f89700b1f90 2023-06-28T20:08:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.530+0000 7f8974a05700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f89700af060 0x7f89700b1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44158/0 (socket says 172.21.15.43:44158) 2023-06-28T20:08:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.530+0000 7f8974a05700 1 -- 172.21.15.43:0/465485327 learned_addr learned my addr 172.21.15.43:0/465485327 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.530+0000 7f8975a07700 1 --2- 172.21.15.43:0/465485327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89700b1f90 0x7f8970160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.530+0000 7f8967fff700 1 -- 172.21.15.43:0/465485327 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630099634 0 0) 0x7f8970160250 con 0x7f89700af060 2023-06-28T20:08:21.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.530+0000 7f8967fff700 1 -- 172.21.15.43:0/465485327 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f895c0035b0 con 0x7f89700af060 2023-06-28T20:08:21.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.531+0000 7f8975a07700 1 -- 172.21.15.43:0/465485327 >> v1:172.21.15.178:6789/0 conn(0x7f89700aec90 legacy=0x7f8970070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:21.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.531+0000 7f8967fff700 1 -- 172.21.15.43:0/465485327 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 947716966 0 0) 0x7f895c0035b0 con 0x7f89700af060 2023-06-28T20:08:21.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.531+0000 7f8975a07700 1 -- 172.21.15.43:0/465485327 >> v1:172.21.15.43:6789/0 conn(0x7f89700af060 legacy=0x7f89700b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:21.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.531+0000 7f8975a07700 1 -- 172.21.15.43:0/465485327 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89701604f0 con 0x7f89700b1f90 2023-06-28T20:08:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.531+0000 7f8975a07700 1 --2- 172.21.15.43:0/465485327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89700b1f90 0x7f8970160cf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8968009dc0 tx=0x7f896800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fc589724471c207 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.531+0000 7f8967fff700 1 -- 172.21.15.43:0/465485327 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896800e070 con 0x7f89700b1f90 2023-06-28T20:08:21.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.531+0000 7f8967fff700 1 -- 172.21.15.43:0/465485327 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8968003a20 con 0x7f89700b1f90 2023-06-28T20:08:21.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.531+0000 7f8967fff700 1 -- 172.21.15.43:0/465485327 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896c001670 con 0x7f89700b1f90 2023-06-28T20:08:21.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.532+0000 7f897746a700 1 -- 172.21.15.43:0/465485327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89700b1f90 msgr2=0x7f8970160cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:21.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.532+0000 7f897746a700 1 --2- 172.21.15.43:0/465485327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89700b1f90 0x7f8970160cf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8968009dc0 tx=0x7f896800b330 comp rx=0 tx=0).stop 2023-06-28T20:08:21.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.532+0000 7f897746a700 1 -- 172.21.15.43:0/465485327 shutdown_connections 2023-06-28T20:08:21.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.532+0000 7f897746a700 1 --2- 172.21.15.43:0/465485327 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89700b1f90 0x7f8970160cf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:21.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.532+0000 7f897746a700 1 -- 172.21.15.43:0/465485327 >> 172.21.15.43:0/465485327 conn(0x7f897006b000 msgr2=0x7f897006b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:21.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.532+0000 7f897746a700 1 -- 172.21.15.43:0/465485327 shutdown_connections 2023-06-28T20:08:21.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.532+0000 7f897746a700 1 -- 172.21.15.43:0/465485327 wait complete. 2023-06-28T20:08:21.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.533+0000 7f897746a700 1 Processor -- start 2023-06-28T20:08:21.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.533+0000 7f897746a700 1 -- start start 2023-06-28T20:08:21.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.534+0000 7f897746a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89700aec90 0x7f897015a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:21.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.534+0000 7f897746a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89700af060 0x7f89700b3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:21.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.534+0000 7f897746a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89700b1f90 0x7f89701f3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:21.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.534+0000 7f897746a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f897015a780 con 0x7f89700aec90 2023-06-28T20:08:21.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.534+0000 7f897746a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89700b3850 con 0x7f89700af060 2023-06-28T20:08:21.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.534+0000 7f897746a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897015cd10 con 0x7f89700b1f90 2023-06-28T20:08:21.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.534+0000 7f8975206700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89700aec90 0x7f897015a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:21.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.534+0000 7f8975206700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89700aec90 0x7f897015a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54978/0 (socket says 172.21.15.43:54978) 2023-06-28T20:08:21.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.534+0000 7f8975206700 1 -- 172.21.15.43:0/1946524049 learned_addr learned my addr 172.21.15.43:0/1946524049 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:21.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.534+0000 7f8974a05700 1 --2- 172.21.15.43:0/1946524049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89700af060 0x7f89700b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:21.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.534+0000 7f8975a07700 1 --2- 172.21.15.43:0/1946524049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89700b1f90 0x7f89701f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:21.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.534+0000 7f8975206700 1 -- 172.21.15.43:0/1946524049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89700b1f90 msgr2=0x7f89701f3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:21.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.534+0000 7f8975206700 1 --2- 172.21.15.43:0/1946524049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89700b1f90 0x7f89701f3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:21.548 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.534+0000 7f8975206700 1 -- 172.21.15.43:0/1946524049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89700af060 msgr2=0x7f89700b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.534+0000 7f8975206700 1 --2- 172.21.15.43:0/1946524049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89700af060 0x7f89700b3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.534+0000 7f8975206700 1 -- 172.21.15.43:0/1946524049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8968009a70 con 0x7f89700aec90 2023-06-28T20:08:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.534+0000 7f8975206700 1 --2- 172.21.15.43:0/1946524049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89700aec90 0x7f897015a240 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f896000c780 tx=0x7f896000cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.534+0000 7f8965ffb700 1 -- 172.21.15.43:0/1946524049 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8960005480 con 0x7f89700aec90 2023-06-28T20:08:21.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.535+0000 7f8965ffb700 1 -- 172.21.15.43:0/1946524049 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8960003680 con 0x7f89700aec90 2023-06-28T20:08:21.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.535+0000 7f897746a700 1 -- 172.21.15.43:0/1946524049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f897015cff0 con 0x7f89700aec90 2023-06-28T20:08:21.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.535+0000 7f8965ffb700 1 -- 172.21.15.43:0/1946524049 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8960003a30 con 0x7f89700aec90 2023-06-28T20:08:21.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.535+0000 7f897746a700 1 -- 172.21.15.43:0/1946524049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897015d5b0 con 0x7f89700aec90 2023-06-28T20:08:21.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.536+0000 7f897746a700 1 -- 172.21.15.43:0/1946524049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89700b4bf0 con 0x7f89700aec90 2023-06-28T20:08:21.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.536+0000 7f8965ffb700 1 -- 172.21.15.43:0/1946524049 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8960003b90 con 0x7f89700aec90 2023-06-28T20:08:21.551 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.537+0000 7f8965ffb700 1 --2- 172.21.15.43:0/1946524049 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f895406ace0 0x7f895406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:21.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.537+0000 7f8965ffb700 1 -- 172.21.15.43:0/1946524049 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8960088ff0 con 0x7f89700aec90 2023-06-28T20:08:21.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.537+0000 7f8974a05700 1 --2- 172.21.15.43:0/1946524049 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f895406ace0 0x7f895406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:21.552 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.537+0000 7f8974a05700 1 --2- 172.21.15.43:0/1946524049 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f895406ace0 0x7f895406d190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f896c001eb0 tx=0x7f896c001c80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.541+0000 7f8965ffb700 1 -- 172.21.15.43:0/1946524049 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8960054060 con 0x7f89700aec90 2023-06-28T20:08:21.789 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:21.786+0000 7f897746a700 1 -- 172.21.15.43:0/1946524049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f89700728b0 con 0x7f89700aec90 2023-06-28T20:08:22.633 INFO:tasks.ceph.mds.a.smithi043.stderr:2023-06-28T20:08:22.629+0000 7f9a71563700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-28T20:08:22.678 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.674+0000 7f8965ffb700 1 -- 172.21.15.43:0/1946524049 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8960057680 con 0x7f89700aec90 2023-06-28T20:08:22.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.677+0000 7f897746a700 1 -- 172.21.15.43:0/1946524049 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f895406ace0 msgr2=0x7f895406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:22.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.678+0000 7f897746a700 1 --2- 172.21.15.43:0/1946524049 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f895406ace0 0x7f895406d190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f896c001eb0 tx=0x7f896c001c80 comp rx=0 tx=0).stop 2023-06-28T20:08:22.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.678+0000 7f897746a700 1 -- 172.21.15.43:0/1946524049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89700aec90 msgr2=0x7f897015a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:22.682 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.678+0000 7f897746a700 1 --2- 172.21.15.43:0/1946524049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89700aec90 0x7f897015a240 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f896000c780 tx=0x7f896000cb40 comp rx=0 tx=0).stop 2023-06-28T20:08:22.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.678+0000 7f897746a700 1 -- 172.21.15.43:0/1946524049 shutdown_connections 2023-06-28T20:08:22.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.678+0000 7f897746a700 1 --2- 172.21.15.43:0/1946524049 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f895406ace0 0x7f895406d190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:22.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.678+0000 7f897746a700 1 --2- 172.21.15.43:0/1946524049 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89700b1f90 0x7f89701f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:22.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.678+0000 7f897746a700 1 --2- 172.21.15.43:0/1946524049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89700aec90 0x7f897015a240 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:22.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.678+0000 7f897746a700 1 --2- 172.21.15.43:0/1946524049 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89700af060 0x7f89700b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:22.683 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.678+0000 7f897746a700 1 -- 172.21.15.43:0/1946524049 >> 172.21.15.43:0/1946524049 conn(0x7f897006b000 msgr2=0x7f897015a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:22.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.678+0000 7f897746a700 1 -- 172.21.15.43:0/1946524049 shutdown_connections 2023-06-28T20:08:22.684 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.678+0000 7f897746a700 1 -- 172.21.15.43:0/1946524049 wait complete. 2023-06-28T20:08:22.696 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-06-28T20:08:22.697 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-06-28T20:08:22.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.828+0000 7f395b52e700 1 Processor -- start 2023-06-28T20:08:22.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.828+0000 7f395b52e700 1 -- start start 2023-06-28T20:08:22.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.828+0000 7f395b52e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39540cf6e0 0x7f39540ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:22.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.829+0000 7f395b52e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f39540cca10 0x7f3954071270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:22.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.829+0000 7f395b52e700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39540717b0 con 0x7f39540b1920 2023-06-28T20:08:22.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.829+0000 7f395b52e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39541609c0 con 0x7f39540cca10 2023-06-28T20:08:22.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.829+0000 7f395b52e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3954160e90 con 0x7f39540cf6e0 2023-06-28T20:08:22.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.829+0000 7f39592ca700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39540cf6e0 0x7f39540ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:22.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.829+0000 7f3958ac9700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f39540cca10 0x7f3954071270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:22.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.829+0000 7f3959acb700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f39540b1920 0x7f39540b1d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:43300/0 (socket says 172.21.15.43:43300) 2023-06-28T20:08:22.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.829+0000 7f3959acb700 1 -- 172.21.15.43:0/3326565540 learned_addr learned my addr 172.21.15.43:0/3326565540 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:22.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.829+0000 7f39592ca700 1 -- 172.21.15.43:0/3326565540 >> v2:172.21.15.178:3300/0 conn(0x7f39540cca10 msgr2=0x7f3954071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:22.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.829+0000 7f39592ca700 1 --2- 172.21.15.43:0/3326565540 >> v2:172.21.15.178:3300/0 conn(0x7f39540cca10 0x7f3954071270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:22.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.829+0000 7f39592ca700 1 -- 172.21.15.43:0/3326565540 >> v1:172.21.15.178:6789/0 conn(0x7f39540b1920 legacy=0x7f39540b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:22.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.829+0000 7f39592ca700 1 -- 172.21.15.43:0/3326565540 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3954160bb0 con 0x7f39540cf6e0 2023-06-28T20:08:22.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.829+0000 7f3958ac9700 1 --2- 172.21.15.43:0/3326565540 >> v2:172.21.15.178:3300/0 conn(0x7f39540cca10 0x7f3954071270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:08:22.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.830+0000 7f39592ca700 1 --2- 172.21.15.43:0/3326565540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39540cf6e0 0x7f39540ce590 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3948009840 tx=0x7f3948009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdf24732027a8e31 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:22.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.830+0000 7f3947fff700 1 -- 172.21.15.43:0/3326565540 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3948011830 con 0x7f39540cf6e0 2023-06-28T20:08:22.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.830+0000 7f3947fff700 1 -- 172.21.15.43:0/3326565540 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3948011990 con 0x7f39540cf6e0 2023-06-28T20:08:22.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.830+0000 7f3947fff700 1 -- 172.21.15.43:0/3326565540 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3948011d20 con 0x7f39540cf6e0 2023-06-28T20:08:22.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.830+0000 7f395b52e700 1 -- 172.21.15.43:0/3326565540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39540cf6e0 msgr2=0x7f39540ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:22.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.830+0000 7f395b52e700 1 --2- 172.21.15.43:0/3326565540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39540cf6e0 0x7f39540ce590 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3948009840 tx=0x7f3948009b50 comp rx=0 tx=0).stop 2023-06-28T20:08:22.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.830+0000 7f395b52e700 1 -- 172.21.15.43:0/3326565540 shutdown_connections 2023-06-28T20:08:22.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.830+0000 7f395b52e700 1 --2- 172.21.15.43:0/3326565540 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39540cf6e0 0x7f39540ce590 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:22.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.830+0000 7f395b52e700 1 --2- 172.21.15.43:0/3326565540 >> v2:172.21.15.178:3300/0 conn(0x7f39540cca10 0x7f3954071270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:22.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.830+0000 7f395b52e700 1 -- 172.21.15.43:0/3326565540 >> 172.21.15.43:0/3326565540 conn(0x7f395406b000 msgr2=0x7f395406b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:22.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.830+0000 7f395b52e700 1 -- 172.21.15.43:0/3326565540 shutdown_connections 2023-06-28T20:08:22.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.830+0000 7f395b52e700 1 -- 172.21.15.43:0/3326565540 wait complete. 2023-06-28T20:08:22.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.832+0000 7f395b52e700 1 Processor -- start 2023-06-28T20:08:22.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.832+0000 7f395b52e700 1 -- start start 2023-06-28T20:08:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.832+0000 7f395b52e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39540b1920 0x7f395415c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:22.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.832+0000 7f395b52e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39540cca10 0x7f39540b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.832+0000 7f395b52e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39540cf6e0 0x7f39540b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.832+0000 7f395b52e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39540b4e80 con 0x7f39540cf6e0 2023-06-28T20:08:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.832+0000 7f395b52e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39540b5160 con 0x7f39540b1920 2023-06-28T20:08:22.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.832+0000 7f395b52e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39540b5440 con 0x7f39540cca10 2023-06-28T20:08:22.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.832+0000 7f3959acb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39540cf6e0 0x7f39540b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:22.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.833+0000 7f3959acb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39540cf6e0 0x7f39540b0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54996/0 (socket says 172.21.15.43:54996) 2023-06-28T20:08:22.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.833+0000 7f3959acb700 1 -- 172.21.15.43:0/186724232 learned_addr learned my addr 172.21.15.43:0/186724232 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:22.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.833+0000 7f3958ac9700 1 --2- 172.21.15.43:0/186724232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39540cca10 0x7f39540b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:22.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.833+0000 7f39592ca700 1 --2- 172.21.15.43:0/186724232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39540b1920 0x7f395415c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:22.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.833+0000 7f3959acb700 1 -- 172.21.15.43:0/186724232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39540cca10 msgr2=0x7f39540b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:22.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.833+0000 7f3959acb700 1 --2- 172.21.15.43:0/186724232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39540cca10 0x7f39540b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:22.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.833+0000 7f3959acb700 1 -- 172.21.15.43:0/186724232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39540b1920 msgr2=0x7f395415c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.833+0000 7f3959acb700 1 --2- 172.21.15.43:0/186724232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39540b1920 0x7f395415c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.833+0000 7f3959acb700 1 -- 172.21.15.43:0/186724232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39500094a0 con 0x7f39540cf6e0 2023-06-28T20:08:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.833+0000 7f3959acb700 1 --2- 172.21.15.43:0/186724232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39540cf6e0 0x7f39540b0b70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f394c00cbc0 tx=0x7f394c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.833+0000 7f3945ffb700 1 -- 172.21.15.43:0/186724232 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c00a9e0 con 0x7f39540cf6e0 2023-06-28T20:08:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.833+0000 7f3945ffb700 1 -- 172.21.15.43:0/186724232 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394c00ab40 con 0x7f39540cf6e0 2023-06-28T20:08:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.833+0000 7f395b52e700 1 -- 172.21.15.43:0/186724232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39480094a0 con 0x7f39540cf6e0 2023-06-28T20:08:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.833+0000 7f3945ffb700 1 -- 172.21.15.43:0/186724232 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c012400 con 0x7f39540cf6e0 2023-06-28T20:08:22.850 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.833+0000 7f395b52e700 1 -- 172.21.15.43:0/186724232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39540b5a00 con 0x7f39540cf6e0 2023-06-28T20:08:22.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.834+0000 7f395b52e700 1 -- 172.21.15.43:0/186724232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f395406cc10 con 0x7f39540cf6e0 2023-06-28T20:08:22.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.835+0000 7f3945ffb700 1 -- 172.21.15.43:0/186724232 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f394c01b3b0 con 0x7f39540cf6e0 2023-06-28T20:08:22.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.835+0000 7f3945ffb700 1 --2- 172.21.15.43:0/186724232 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f393806ace0 0x7f393806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:22.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.835+0000 7f3945ffb700 1 -- 172.21.15.43:0/186724232 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f394c088f20 con 0x7f39540cf6e0 2023-06-28T20:08:22.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.835+0000 7f39592ca700 1 --2- 172.21.15.43:0/186724232 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f393806ace0 0x7f393806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:22.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.835+0000 7f39592ca700 1 --2- 172.21.15.43:0/186724232 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f393806ace0 0x7f393806d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3948000c00 tx=0x7f3948008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:22.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:22.839+0000 7f3945ffb700 1 -- 172.21.15.43:0/186724232 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f395406cc10 con 0x7f39540cf6e0 2023-06-28T20:08:23.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:23.084+0000 7f395b52e700 1 -- 172.21.15.43:0/186724232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f39541e5e20 con 0x7f39540cf6e0 2023-06-28T20:08:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:23.916+0000 7f3945ffb700 1 -- 172.21.15.43:0/186724232 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7f39541e5e20 con 0x7f39540cf6e0 2023-06-28T20:08:23.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:23.919+0000 7f395b52e700 1 -- 172.21.15.43:0/186724232 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f393806ace0 msgr2=0x7f393806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:23.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:23.919+0000 7f395b52e700 1 --2- 172.21.15.43:0/186724232 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f393806ace0 0x7f393806d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3948000c00 tx=0x7f3948008030 comp rx=0 tx=0).stop 2023-06-28T20:08:23.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:23.919+0000 7f395b52e700 1 -- 172.21.15.43:0/186724232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39540cf6e0 msgr2=0x7f39540b0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:23.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:23.919+0000 7f395b52e700 1 --2- 172.21.15.43:0/186724232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39540cf6e0 0x7f39540b0b70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f394c00cbc0 tx=0x7f394c00ced0 comp rx=0 tx=0).stop 2023-06-28T20:08:23.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:23.919+0000 7f395b52e700 1 -- 172.21.15.43:0/186724232 shutdown_connections 2023-06-28T20:08:23.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:23.919+0000 7f395b52e700 1 --2- 172.21.15.43:0/186724232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39540cf6e0 0x7f39540b0b70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:23.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:23.919+0000 7f395b52e700 1 --2- 172.21.15.43:0/186724232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39540b1920 0x7f395415c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:23.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:23.919+0000 7f395b52e700 1 --2- 172.21.15.43:0/186724232 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f393806ace0 0x7f393806d190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:23.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:23.919+0000 7f395b52e700 1 --2- 172.21.15.43:0/186724232 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39540cca10 0x7f39540b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:23.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:23.919+0000 7f395b52e700 1 -- 172.21.15.43:0/186724232 >> 172.21.15.43:0/186724232 conn(0x7f395406b000 msgr2=0x7f39540cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:23.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:23.919+0000 7f395b52e700 1 -- 172.21.15.43:0/186724232 shutdown_connections 2023-06-28T20:08:23.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:23.919+0000 7f395b52e700 1 -- 172.21.15.43:0/186724232 wait complete. 2023-06-28T20:08:23.938 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-06-28T20:08:23.939 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-28T20:08:24.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.068+0000 7f961a757700 1 Processor -- start 2023-06-28T20:08:24.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.068+0000 7f961a757700 1 -- start start 2023-06-28T20:08:24.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f961a757700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96140cc600 0x7f961415a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:24.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f961a757700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f961406ee20 0x7f96140cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:24.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f961a757700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96140ce0a0 con 0x7f9614159820 2023-06-28T20:08:24.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f961a757700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96140ce380 con 0x7f961406ee20 2023-06-28T20:08:24.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f961a757700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9614159c90 con 0x7f96140cc600 2023-06-28T20:08:24.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f96137fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9614159820 0x7f96140cc9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44186/0 (socket says 172.21.15.43:44186) 2023-06-28T20:08:24.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f96137fe700 1 -- 172.21.15.43:0/3488206241 learned_addr learned my addr 172.21.15.43:0/3488206241 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:24.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f9613fff700 1 --2- 172.21.15.43:0/3488206241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96140cc600 0x7f961415a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:24.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f9618cf4700 1 --2- 172.21.15.43:0/3488206241 >> v2:172.21.15.178:3300/0 conn(0x7f961406ee20 0x7f96140cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:24.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f9612ffd700 1 -- 172.21.15.43:0/3488206241 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2465263135 0 0) 0x7f96140ce0a0 con 0x7f9614159820 2023-06-28T20:08:24.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f9612ffd700 1 -- 172.21.15.43:0/3488206241 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95fc0035b0 con 0x7f9614159820 2023-06-28T20:08:24.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f9612ffd700 1 -- 172.21.15.43:0/3488206241 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3987649397 0 0) 0x7f95fc0035b0 con 0x7f9614159820 2023-06-28T20:08:24.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f9612ffd700 1 -- 172.21.15.43:0/3488206241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96140cc600 msgr2=0x7f961415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:24.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f9612ffd700 1 --2- 172.21.15.43:0/3488206241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96140cc600 0x7f961415a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f9612ffd700 1 -- 172.21.15.43:0/3488206241 >> v2:172.21.15.178:3300/0 conn(0x7f961406ee20 msgr2=0x7f96140cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:24.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f9612ffd700 1 --2- 172.21.15.43:0/3488206241 >> v2:172.21.15.178:3300/0 conn(0x7f961406ee20 0x7f96140cd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f9612ffd700 1 -- 172.21.15.43:0/3488206241 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9614159f70 con 0x7f9614159820 2023-06-28T20:08:24.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f9612ffd700 1 -- 172.21.15.43:0/3488206241 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f9608001670 con 0x7f9614159820 2023-06-28T20:08:24.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f9612ffd700 1 -- 172.21.15.43:0/3488206241 >> v1:172.21.15.43:6789/0 conn(0x7f9614159820 legacy=0x7f96140cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:24.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f9612ffd700 1 --2- 172.21.15.43:0/3488206241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc003a00 0x7f95fc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:24.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f9612ffd700 1 --2- 172.21.15.43:0/3488206241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95fc004430 0x7f95fc0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:24.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f9612ffd700 1 --2- 172.21.15.43:0/3488206241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96140cc600 0x7f95fc004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:24.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f9612ffd700 1 -- 172.21.15.43:0/3488206241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96140ce380 con 0x7f95fc003a00 2023-06-28T20:08:24.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f9612ffd700 1 -- 172.21.15.43:0/3488206241 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9614159c90 con 0x7f95fc004430 2023-06-28T20:08:24.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f9612ffd700 1 -- 172.21.15.43:0/3488206241 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc00ba10 con 0x7f96140cc600 2023-06-28T20:08:24.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.069+0000 7f9613fff700 1 --2- 172.21.15.43:0/3488206241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc003a00 0x7f95fc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:24.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.070+0000 7f96137fe700 1 --2- 172.21.15.43:0/3488206241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95fc004430 0x7f95fc0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:24.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.070+0000 7f9618cf4700 1 --2- 172.21.15.43:0/3488206241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96140cc600 0x7f95fc004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:24.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.070+0000 7f9613fff700 1 -- 172.21.15.43:0/3488206241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96140cc600 msgr2=0x7f95fc004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:24.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.070+0000 7f9613fff700 1 --2- 172.21.15.43:0/3488206241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96140cc600 0x7f95fc004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.070+0000 7f9613fff700 1 -- 172.21.15.43:0/3488206241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95fc004430 msgr2=0x7f95fc0048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:24.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.070+0000 7f9613fff700 1 --2- 172.21.15.43:0/3488206241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95fc004430 0x7f95fc0048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.070+0000 7f9613fff700 1 -- 172.21.15.43:0/3488206241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc0035b0 con 0x7f95fc003a00 2023-06-28T20:08:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.070+0000 7f9618cf4700 1 --2- 172.21.15.43:0/3488206241 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96140cc600 0x7f95fc004fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.070+0000 7f9613fff700 1 --2- 172.21.15.43:0/3488206241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc003a00 0x7f95fc003e10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9604009b60 tx=0x7f95fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35a2e99ddecb99e server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.070+0000 7f9612ffd700 1 -- 172.21.15.43:0/3488206241 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960400e6c0 con 0x7f95fc003a00 2023-06-28T20:08:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.070+0000 7f9612ffd700 1 -- 172.21.15.43:0/3488206241 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960400e820 con 0x7f95fc003a00 2023-06-28T20:08:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.070+0000 7f9612ffd700 1 -- 172.21.15.43:0/3488206241 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960400eb90 con 0x7f95fc003a00 2023-06-28T20:08:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.071+0000 7f961a757700 1 -- 172.21.15.43:0/3488206241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc003a00 msgr2=0x7f95fc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:24.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.071+0000 7f961a757700 1 --2- 172.21.15.43:0/3488206241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc003a00 0x7f95fc003e10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9604009b60 tx=0x7f95fc005880 comp rx=0 tx=0).stop 2023-06-28T20:08:24.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.071+0000 7f9618cf4700 1 -- 172.21.15.43:0/3488206241 reap_dead start 2023-06-28T20:08:24.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.071+0000 7f961a757700 1 -- 172.21.15.43:0/3488206241 shutdown_connections 2023-06-28T20:08:24.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.071+0000 7f961a757700 1 -- 172.21.15.43:0/3488206241 >> 172.21.15.43:0/3488206241 conn(0x7f96140b1720 msgr2=0x7f96140b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:24.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.071+0000 7f961a757700 1 -- 172.21.15.43:0/3488206241 shutdown_connections 2023-06-28T20:08:24.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.071+0000 7f961a757700 1 -- 172.21.15.43:0/3488206241 wait complete. 2023-06-28T20:08:24.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.072+0000 7f961a757700 1 Processor -- start 2023-06-28T20:08:24.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.072+0000 7f961a757700 1 -- start start 2023-06-28T20:08:24.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.073+0000 7f961a757700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc003a00 0x7f9614147020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:24.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.073+0000 7f961a757700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96140c6440 0x7f96141f2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:24.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.073+0000 7f961a757700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f961415a9f0 0x7f961415ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:24.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.073+0000 7f961a757700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961415b3a0 con 0x7f95fc003a00 2023-06-28T20:08:24.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.073+0000 7f961a757700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96140c7050 con 0x7f961415a9f0 2023-06-28T20:08:24.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.073+0000 7f961a757700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9614147560 con 0x7f96140c6440 2023-06-28T20:08:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.073+0000 7f9613fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc003a00 0x7f9614147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.073+0000 7f9613fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc003a00 0x7f9614147020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55012/0 (socket says 172.21.15.43:55012) 2023-06-28T20:08:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.073+0000 7f9613fff700 1 -- 172.21.15.43:0/2747599865 learned_addr learned my addr 172.21.15.43:0/2747599865 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.073+0000 7f96137fe700 1 --2- 172.21.15.43:0/2747599865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96140c6440 0x7f96141f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.073+0000 7f9618cf4700 1 --2- 172.21.15.43:0/2747599865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f961415a9f0 0x7f961415ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.073+0000 7f9613fff700 1 -- 172.21.15.43:0/2747599865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96140c6440 msgr2=0x7f96141f2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.073+0000 7f9613fff700 1 --2- 172.21.15.43:0/2747599865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96140c6440 0x7f96141f2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.073+0000 7f9613fff700 1 -- 172.21.15.43:0/2747599865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f961415a9f0 msgr2=0x7f961415ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:24.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.073+0000 7f9613fff700 1 --2- 172.21.15.43:0/2747599865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f961415a9f0 0x7f961415ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.073+0000 7f9613fff700 1 -- 172.21.15.43:0/2747599865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96141f30a0 con 0x7f95fc003a00 2023-06-28T20:08:24.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.073+0000 7f96137fe700 1 --2- 172.21.15.43:0/2747599865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96140c6440 0x7f96141f2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:24.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.073+0000 7f9618cf4700 1 --2- 172.21.15.43:0/2747599865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f961415a9f0 0x7f961415ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:24.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.074+0000 7f9613fff700 1 --2- 172.21.15.43:0/2747599865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc003a00 0x7f9614147020 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0056d0 tx=0x7f96040047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:24.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.074+0000 7f9610ff9700 1 -- 172.21.15.43:0/2747599865 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960400e6c0 con 0x7f95fc003a00 2023-06-28T20:08:24.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.074+0000 7f9610ff9700 1 -- 172.21.15.43:0/2747599865 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9604003ab0 con 0x7f95fc003a00 2023-06-28T20:08:24.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.074+0000 7f961a757700 1 -- 172.21.15.43:0/2747599865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96141f3380 con 0x7f95fc003a00 2023-06-28T20:08:24.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.074+0000 7f9610ff9700 1 -- 172.21.15.43:0/2747599865 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9604003e60 con 0x7f95fc003a00 2023-06-28T20:08:24.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.074+0000 7f961a757700 1 -- 172.21.15.43:0/2747599865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96140c6ce0 con 0x7f95fc003a00 2023-06-28T20:08:24.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.075+0000 7f961a757700 1 -- 172.21.15.43:0/2747599865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96140c7330 con 0x7f95fc003a00 2023-06-28T20:08:24.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.075+0000 7f9610ff9700 1 -- 172.21.15.43:0/2747599865 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f960401e660 con 0x7f95fc003a00 2023-06-28T20:08:24.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.075+0000 7f9610ff9700 1 --2- 172.21.15.43:0/2747599865 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f960006ace0 0x7f960006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:24.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.076+0000 7f9610ff9700 1 -- 172.21.15.43:0/2747599865 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f960408d050 con 0x7f95fc003a00 2023-06-28T20:08:24.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.076+0000 7f96137fe700 1 --2- 172.21.15.43:0/2747599865 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f960006ace0 0x7f960006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:24.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.076+0000 7f96137fe700 1 --2- 172.21.15.43:0/2747599865 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f960006ace0 0x7f960006d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9608000fb0 tx=0x7f9608009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:24.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.080+0000 7f9610ff9700 1 -- 172.21.15.43:0/2747599865 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f96040582b0 con 0x7f95fc003a00 2023-06-28T20:08:24.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.325+0000 7f961a757700 1 -- 172.21.15.43:0/2747599865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96140c6ea0 con 0x7f95fc003a00 2023-06-28T20:08:24.329 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.325+0000 7f9610ff9700 1 -- 172.21.15.43:0/2747599865 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f960405b8d0 con 0x7f95fc003a00 2023-06-28T20:08:24.330 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:24.330 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1274138939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":1274138939},{"type":"v1","addr":"172.21.15.178:6835","nonce":1274138939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3882460026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3882460026},{"type":"v1","addr":"172.21.15.178:6834","nonce":3882460026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/382468709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":382468709},{"type":"v1","addr":"172.21.15.43:6836","nonce":382468709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-28T20:08:20.073546+0000","modified":"2023-06-28T20:08:23.914623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5112},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5112":{"gid":5112,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6837/1096784002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":1096784002},{"type":"v1","addr":"172.21.15.43:6837","nonce":1096784002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-06-28T20:08:24.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.327+0000 7f961a757700 1 -- 172.21.15.43:0/2747599865 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f960006ace0 msgr2=0x7f960006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:24.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.327+0000 7f961a757700 1 --2- 172.21.15.43:0/2747599865 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f960006ace0 0x7f960006d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9608000fb0 tx=0x7f9608009040 comp rx=0 tx=0).stop 2023-06-28T20:08:24.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.327+0000 7f961a757700 1 -- 172.21.15.43:0/2747599865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc003a00 msgr2=0x7f9614147020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:24.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.327+0000 7f961a757700 1 --2- 172.21.15.43:0/2747599865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc003a00 0x7f9614147020 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0056d0 tx=0x7f96040047c0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.327+0000 7f961a757700 1 -- 172.21.15.43:0/2747599865 shutdown_connections 2023-06-28T20:08:24.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.327+0000 7f961a757700 1 --2- 172.21.15.43:0/2747599865 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f961415a9f0 0x7f961415ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.327+0000 7f961a757700 1 --2- 172.21.15.43:0/2747599865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc003a00 0x7f9614147020 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.327+0000 7f961a757700 1 --2- 172.21.15.43:0/2747599865 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f960006ace0 0x7f960006d190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.327+0000 7f961a757700 1 --2- 172.21.15.43:0/2747599865 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f96140c6440 0x7f96141f2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.327+0000 7f961a757700 1 -- 172.21.15.43:0/2747599865 >> 172.21.15.43:0/2747599865 conn(0x7f96140b1720 msgr2=0x7f96140ca220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:24.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.328+0000 7f961a757700 1 -- 172.21.15.43:0/2747599865 shutdown_connections 2023-06-28T20:08:24.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.328+0000 7f961a757700 1 -- 172.21.15.43:0/2747599865 wait complete. 2023-06-28T20:08:24.334 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 15 2023-06-28T20:08:24.344 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:08:24.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.469+0000 7f6983b2c700 1 Processor -- start 2023-06-28T20:08:24.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.469+0000 7f6983b2c700 1 -- start start 2023-06-28T20:08:24.476 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.470+0000 7f6983b2c700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f697c0cc600 0x7f697c15a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:24.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.470+0000 7f6983b2c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697c0ce0a0 con 0x7f697c159820 2023-06-28T20:08:24.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.470+0000 7f6983b2c700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697c0ce380 con 0x7f697c06ee20 2023-06-28T20:08:24.477 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.470+0000 7f6983b2c700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f697c159c90 con 0x7f697c0cc600 2023-06-28T20:08:24.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.470+0000 7f69810c7700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f697c159820 0x7f697c0cc9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44206/0 (socket says 172.21.15.43:44206) 2023-06-28T20:08:24.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.470+0000 7f69810c7700 1 -- 172.21.15.43:0/2915432785 learned_addr learned my addr 172.21.15.43:0/2915432785 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:24.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.470+0000 7f69818c8700 1 --2- 172.21.15.43:0/2915432785 >> v2:172.21.15.178:3300/0 conn(0x7f697c0cc600 0x7f697c15a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:24.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.470+0000 7f69808c6700 1 -- 172.21.15.43:0/2915432785 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4032811552 0 0) 0x7f697c0ce0a0 con 0x7f697c159820 2023-06-28T20:08:24.478 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.470+0000 7f69808c6700 1 -- 172.21.15.43:0/2915432785 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69680057c0 con 0x7f697c159820 2023-06-28T20:08:24.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.470+0000 7f69808c6700 1 -- 172.21.15.43:0/2915432785 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3913801395 0 0) 0x7f69680057c0 con 0x7f697c159820 2023-06-28T20:08:24.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.470+0000 7f69808c6700 1 -- 172.21.15.43:0/2915432785 >> v2:172.21.15.178:3300/0 conn(0x7f697c0cc600 msgr2=0x7f697c15a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:24.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.470+0000 7f69808c6700 1 --2- 172.21.15.43:0/2915432785 >> v2:172.21.15.178:3300/0 conn(0x7f697c0cc600 0x7f697c15a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.470+0000 7f69808c6700 1 -- 172.21.15.43:0/2915432785 >> v1:172.21.15.178:6789/0 conn(0x7f697c06ee20 legacy=0x7f697c0cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:24.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.471+0000 7f69808c6700 1 -- 172.21.15.43:0/2915432785 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c159f70 con 0x7f697c159820 2023-06-28T20:08:24.479 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.471+0000 7f69808c6700 1 -- 172.21.15.43:0/2915432785 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f696c001670 con 0x7f697c159820 2023-06-28T20:08:24.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.471+0000 7f69808c6700 1 -- 172.21.15.43:0/2915432785 >> v1:172.21.15.43:6789/0 conn(0x7f697c159820 legacy=0x7f697c0cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:24.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.471+0000 7f69808c6700 1 --2- 172.21.15.43:0/2915432785 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6968005c10 0x7f6968006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:24.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.471+0000 7f69808c6700 1 --2- 172.21.15.43:0/2915432785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6968006a60 0x7f6968006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:24.480 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.471+0000 7f69808c6700 1 --2- 172.21.15.43:0/2915432785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6968007410 0x7f696800dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:24.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.471+0000 7f69808c6700 1 -- 172.21.15.43:0/2915432785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c159c90 con 0x7f6968006a60 2023-06-28T20:08:24.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.471+0000 7f69808c6700 1 -- 172.21.15.43:0/2915432785 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696800e290 con 0x7f6968007410 2023-06-28T20:08:24.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.471+0000 7f69808c6700 1 -- 172.21.15.43:0/2915432785 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f696800e450 con 0x7f6968005c10 2023-06-28T20:08:24.481 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.471+0000 7f69810c7700 1 --2- 172.21.15.43:0/2915432785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6968006a60 0x7f6968006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:24.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.471+0000 7f69818c8700 1 --2- 172.21.15.43:0/2915432785 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6968005c10 0x7f6968006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:24.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.471+0000 7f69820c9700 1 --2- 172.21.15.43:0/2915432785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6968007410 0x7f696800dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:24.482 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.471+0000 7f69810c7700 1 -- 172.21.15.43:0/2915432785 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6968005c10 msgr2=0x7f6968006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:24.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.471+0000 7f69810c7700 1 --2- 172.21.15.43:0/2915432785 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6968005c10 0x7f6968006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.471+0000 7f69810c7700 1 -- 172.21.15.43:0/2915432785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6968007410 msgr2=0x7f696800dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:24.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.471+0000 7f69810c7700 1 --2- 172.21.15.43:0/2915432785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6968007410 0x7f696800dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.471+0000 7f69810c7700 1 -- 172.21.15.43:0/2915432785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69680057c0 con 0x7f6968006a60 2023-06-28T20:08:24.483 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.471+0000 7f69820c9700 1 --2- 172.21.15.43:0/2915432785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6968007410 0x7f696800dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:24.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.471+0000 7f69820c9700 1 -- 172.21.15.43:0/2915432785 reap_dead start 2023-06-28T20:08:24.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.471+0000 7f69810c7700 1 --2- 172.21.15.43:0/2915432785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6968006a60 0x7f6968006ed0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f696c00c240 tx=0x7f6968007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eea39f6ab5f3c4c server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:24.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.471+0000 7f69808c6700 1 -- 172.21.15.43:0/2915432785 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c007c10 con 0x7f6968006a60 2023-06-28T20:08:24.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.471+0000 7f69808c6700 1 -- 172.21.15.43:0/2915432785 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696c00cde0 con 0x7f6968006a60 2023-06-28T20:08:24.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.471+0000 7f69808c6700 1 -- 172.21.15.43:0/2915432785 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696c005ba0 con 0x7f6968006a60 2023-06-28T20:08:24.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.472+0000 7f6983b2c700 1 -- 172.21.15.43:0/2915432785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6968006a60 msgr2=0x7f6968006ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:24.484 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.472+0000 7f6983b2c700 1 --2- 172.21.15.43:0/2915432785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6968006a60 0x7f6968006ed0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f696c00c240 tx=0x7f6968007a90 comp rx=0 tx=0).stop 2023-06-28T20:08:24.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.472+0000 7f6983b2c700 1 -- 172.21.15.43:0/2915432785 shutdown_connections 2023-06-28T20:08:24.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.472+0000 7f6983b2c700 1 --2- 172.21.15.43:0/2915432785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6968006a60 0x7f6968006ed0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.472+0000 7f6983b2c700 1 -- 172.21.15.43:0/2915432785 >> 172.21.15.43:0/2915432785 conn(0x7f697c0b1720 msgr2=0x7f697c0b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:24.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.472+0000 7f6983b2c700 1 -- 172.21.15.43:0/2915432785 shutdown_connections 2023-06-28T20:08:24.485 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.472+0000 7f6983b2c700 1 -- 172.21.15.43:0/2915432785 wait complete. 2023-06-28T20:08:24.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.473+0000 7f6983b2c700 1 Processor -- start 2023-06-28T20:08:24.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.473+0000 7f6983b2c700 1 -- start start 2023-06-28T20:08:24.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.474+0000 7f6983b2c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6968006a60 0x7f697c150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:24.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.474+0000 7f6983b2c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f697c1f56e0 0x7f697c1f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:24.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.474+0000 7f6983b2c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f697c15aa30 0x7f697c15aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:24.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.474+0000 7f6983b2c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c0b3d70 con 0x7f6968006a60 2023-06-28T20:08:24.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.474+0000 7f6983b2c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c15b3f0 con 0x7f697c1f56e0 2023-06-28T20:08:24.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.474+0000 7f6983b2c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697c1f6180 con 0x7f697c15aa30 2023-06-28T20:08:24.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.474+0000 7f69818c8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6968006a60 0x7f697c150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:24.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.474+0000 7f69818c8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6968006a60 0x7f697c150040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55032/0 (socket says 172.21.15.43:55032) 2023-06-28T20:08:24.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.474+0000 7f69818c8700 1 -- 172.21.15.43:0/1199693427 learned_addr learned my addr 172.21.15.43:0/1199693427 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:24.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.474+0000 7f69820c9700 1 --2- 172.21.15.43:0/1199693427 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f697c15aa30 0x7f697c15aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:24.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.474+0000 7f69810c7700 1 --2- 172.21.15.43:0/1199693427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f697c1f56e0 0x7f697c1f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:24.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.474+0000 7f69818c8700 1 -- 172.21.15.43:0/1199693427 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f697c15aa30 msgr2=0x7f697c15aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:24.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.474+0000 7f69818c8700 1 --2- 172.21.15.43:0/1199693427 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f697c15aa30 0x7f697c15aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.490 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.474+0000 7f69818c8700 1 -- 172.21.15.43:0/1199693427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f697c1f56e0 msgr2=0x7f697c1f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:24.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.474+0000 7f69818c8700 1 --2- 172.21.15.43:0/1199693427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f697c1f56e0 0x7f697c1f5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.474+0000 7f69818c8700 1 -- 172.21.15.43:0/1199693427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c1f6460 con 0x7f6968006a60 2023-06-28T20:08:24.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.474+0000 7f69810c7700 1 --2- 172.21.15.43:0/1199693427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f697c1f56e0 0x7f697c1f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:24.491 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.474+0000 7f69820c9700 1 --2- 172.21.15.43:0/1199693427 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f697c15aa30 0x7f697c15aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:24.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.475+0000 7f69818c8700 1 --2- 172.21.15.43:0/1199693427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6968006a60 0x7f697c150040 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6978011aa0 tx=0x7f6978011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:24.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.475+0000 7f69727fc700 1 -- 172.21.15.43:0/1199693427 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697800f9e0 con 0x7f6968006a60 2023-06-28T20:08:24.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.475+0000 7f69727fc700 1 -- 172.21.15.43:0/1199693427 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697800fb40 con 0x7f6968006a60 2023-06-28T20:08:24.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.475+0000 7f6983b2c700 1 -- 172.21.15.43:0/1199693427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697c0b3f80 con 0x7f6968006a60 2023-06-28T20:08:24.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.475+0000 7f69727fc700 1 -- 172.21.15.43:0/1199693427 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6978014440 con 0x7f6968006a60 2023-06-28T20:08:24.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.475+0000 7f6983b2c700 1 -- 172.21.15.43:0/1199693427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697c1f2410 con 0x7f6968006a60 2023-06-28T20:08:24.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.476+0000 7f6983b2c700 1 -- 172.21.15.43:0/1199693427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697c0b44a0 con 0x7f6968006a60 2023-06-28T20:08:24.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.476+0000 7f69727fc700 1 -- 172.21.15.43:0/1199693427 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f69780145a0 con 0x7f6968006a60 2023-06-28T20:08:24.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.477+0000 7f69727fc700 1 --2- 172.21.15.43:0/1199693427 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f696006ace0 0x7f696006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:24.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.477+0000 7f69727fc700 1 -- 172.21.15.43:0/1199693427 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6978089ca0 con 0x7f6968006a60 2023-06-28T20:08:24.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.477+0000 7f69810c7700 1 --2- 172.21.15.43:0/1199693427 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f696006ace0 0x7f696006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:24.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.477+0000 7f69810c7700 1 --2- 172.21.15.43:0/1199693427 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f696006ace0 0x7f696006d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f696c001930 tx=0x7f696c007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:24.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.481+0000 7f69727fc700 1 -- 172.21.15.43:0/1199693427 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f697c0b44a0 con 0x7f6968006a60 2023-06-28T20:08:24.678 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-28T20:08:24.681 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-28T20:08:24.685 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.682+0000 7f6983b2c700 1 -- 172.21.15.43:0/1199693427 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f697c0728b0 con 0x7f6968006a60 2023-06-28T20:08:24.686 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.683+0000 7f69727fc700 1 -- 172.21.15.43:0/1199693427 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+18293 (secure 0 0 0) 0x7f6978054d50 con 0x7f6968006a60 2023-06-28T20:08:24.687 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:24.687 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":35,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:08:21.654904+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:08:17.811454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-06-28T20:08:19.046351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:08:24.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.685+0000 7f6983b2c700 1 -- 172.21.15.43:0/1199693427 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f696006ace0 msgr2=0x7f696006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:24.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.685+0000 7f6983b2c700 1 --2- 172.21.15.43:0/1199693427 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f696006ace0 0x7f696006d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f696c001930 tx=0x7f696c007000 comp rx=0 tx=0).stop 2023-06-28T20:08:24.693 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.685+0000 7f6983b2c700 1 -- 172.21.15.43:0/1199693427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6968006a60 msgr2=0x7f697c150040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:24.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.685+0000 7f6983b2c700 1 --2- 172.21.15.43:0/1199693427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6968006a60 0x7f697c150040 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6978011aa0 tx=0x7f6978011e60 comp rx=0 tx=0).stop 2023-06-28T20:08:24.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.685+0000 7f6983b2c700 1 -- 172.21.15.43:0/1199693427 shutdown_connections 2023-06-28T20:08:24.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.685+0000 7f6983b2c700 1 --2- 172.21.15.43:0/1199693427 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f696006ace0 0x7f696006d190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.685+0000 7f6983b2c700 1 --2- 172.21.15.43:0/1199693427 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f697c15aa30 0x7f697c15aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.685+0000 7f6983b2c700 1 --2- 172.21.15.43:0/1199693427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6968006a60 0x7f697c150040 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.685+0000 7f6983b2c700 1 --2- 172.21.15.43:0/1199693427 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f697c1f56e0 0x7f697c1f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.685+0000 7f6983b2c700 1 -- 172.21.15.43:0/1199693427 >> 172.21.15.43:0/1199693427 conn(0x7f697c0b1720 msgr2=0x7f697c0d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:24.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.685+0000 7f6983b2c700 1 -- 172.21.15.43:0/1199693427 shutdown_connections 2023-06-28T20:08:24.695 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.685+0000 7f6983b2c700 1 -- 172.21.15.43:0/1199693427 wait complete. 2023-06-28T20:08:24.703 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-06-28T20:08:24.703 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-28T20:08:24.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.898+0000 7efea3ce2700 1 Processor -- start 2023-06-28T20:08:24.902 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.898+0000 7efea3ce2700 1 -- start start 2023-06-28T20:08:24.905 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.899+0000 7efea3ce2700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe9c0cbd40 0x7efe9c06b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.899+0000 7efea3ce2700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efe9c06b540 0x7efe9c0ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.899+0000 7efea3ce2700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe9c0ccf10 con 0x7efe9c0d0630 2023-06-28T20:08:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.899+0000 7efea3ce2700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe9c06b9c0 con 0x7efe9c06b540 2023-06-28T20:08:24.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.899+0000 7efea3ce2700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c15cd80 con 0x7efe9c0cbd40 2023-06-28T20:08:24.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.899+0000 7efea127d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe9c0cbd40 0x7efe9c06b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:24.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.899+0000 7efea227f700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efe9c06b540 0x7efe9c0ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:24.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.899+0000 7efea127d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe9c0cbd40 0x7efe9c06b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.43:39014/0 (socket says 172.21.15.43:39014) 2023-06-28T20:08:24.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.899+0000 7efea227f700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7efe9c06b540 0x7efe9c0ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:41108/0 (socket says 172.21.15.43:41108) 2023-06-28T20:08:24.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.899+0000 7efea127d700 1 -- 172.21.15.43:0/1869797562 learned_addr learned my addr 172.21.15.43:0/1869797562 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:24.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.899+0000 7efea1a7e700 1 --1- 172.21.15.43:0/1869797562 >> v1:172.21.15.178:6789/0 conn(0x7efe9c0d0630 0x7efe9c0d1070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:43350/0 (socket says 172.21.15.43:43350) 2023-06-28T20:08:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.899+0000 7efea127d700 1 -- 172.21.15.43:0/1869797562 >> v2:172.21.15.178:3300/0 conn(0x7efe9c06b540 msgr2=0x7efe9c0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.899+0000 7efea127d700 1 --2- 172.21.15.43:0/1869797562 >> v2:172.21.15.178:3300/0 conn(0x7efe9c06b540 0x7efe9c0ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.899+0000 7efea127d700 1 -- 172.21.15.43:0/1869797562 >> v1:172.21.15.178:6789/0 conn(0x7efe9c0d0630 legacy=0x7efe9c0d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.900+0000 7efea127d700 1 -- 172.21.15.43:0/1869797562 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9c0d0a00 con 0x7efe9c0cbd40 2023-06-28T20:08:24.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.900+0000 7efea227f700 1 --2- 172.21.15.43:0/1869797562 >> v2:172.21.15.178:3300/0 conn(0x7efe9c06b540 0x7efe9c0ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:08:24.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.900+0000 7efea127d700 1 --2- 172.21.15.43:0/1869797562 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe9c0cbd40 0x7efe9c06b000 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efe8c0098f0 tx=0x7efe8c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6601037001430df server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:24.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.900+0000 7efea0a7c700 1 -- 172.21.15.43:0/1869797562 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c011830 con 0x7efe9c0cbd40 2023-06-28T20:08:24.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.900+0000 7efea0a7c700 1 -- 172.21.15.43:0/1869797562 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8c011990 con 0x7efe9c0cbd40 2023-06-28T20:08:24.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.900+0000 7efea0a7c700 1 -- 172.21.15.43:0/1869797562 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c011d20 con 0x7efe9c0cbd40 2023-06-28T20:08:24.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.900+0000 7efea3ce2700 1 -- 172.21.15.43:0/1869797562 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe9c0cbd40 msgr2=0x7efe9c06b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:24.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.900+0000 7efea3ce2700 1 --2- 172.21.15.43:0/1869797562 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe9c0cbd40 0x7efe9c06b000 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efe8c0098f0 tx=0x7efe8c009c00 comp rx=0 tx=0).stop 2023-06-28T20:08:24.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.900+0000 7efea3ce2700 1 -- 172.21.15.43:0/1869797562 shutdown_connections 2023-06-28T20:08:24.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.900+0000 7efea3ce2700 1 --2- 172.21.15.43:0/1869797562 >> v2:172.21.15.178:3300/0 conn(0x7efe9c06b540 0x7efe9c0ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.900+0000 7efea3ce2700 1 --2- 172.21.15.43:0/1869797562 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe9c0cbd40 0x7efe9c06b000 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.900+0000 7efea3ce2700 1 -- 172.21.15.43:0/1869797562 >> 172.21.15.43:0/1869797562 conn(0x7efe9c0664b0 msgr2=0x7efe9c09a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:24.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.901+0000 7efea3ce2700 1 -- 172.21.15.43:0/1869797562 shutdown_connections 2023-06-28T20:08:24.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.901+0000 7efea3ce2700 1 -- 172.21.15.43:0/1869797562 wait complete. 2023-06-28T20:08:24.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.902+0000 7efea3ce2700 1 Processor -- start 2023-06-28T20:08:24.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.902+0000 7efea3ce2700 1 -- start start 2023-06-28T20:08:24.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.902+0000 7efea3ce2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe9c06b540 0x7efe9c0b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:24.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.902+0000 7efea3ce2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe9c0cbd40 0x7efe9c0b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:24.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.902+0000 7efea3ce2700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe9c0d0630 0x7efe9c0c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:24.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.902+0000 7efea3ce2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c0b23e0 con 0x7efe9c06b540 2023-06-28T20:08:24.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.902+0000 7efea3ce2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c0b17d0 con 0x7efe9c0cbd40 2023-06-28T20:08:24.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.902+0000 7efea3ce2700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c0b1ab0 con 0x7efe9c0d0630 2023-06-28T20:08:24.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.903+0000 7efea1a7e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe9c06b540 0x7efe9c0b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:24.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.903+0000 7efea1a7e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe9c06b540 0x7efe9c0b11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55046/0 (socket says 172.21.15.43:55046) 2023-06-28T20:08:24.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.903+0000 7efea1a7e700 1 -- 172.21.15.43:0/3786629944 learned_addr learned my addr 172.21.15.43:0/3786629944 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:24.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.903+0000 7efea227f700 1 --2- 172.21.15.43:0/3786629944 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe9c0d0630 0x7efe9c0c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:24.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.903+0000 7efea127d700 1 --2- 172.21.15.43:0/3786629944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe9c0cbd40 0x7efe9c0b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:24.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.903+0000 7efea1a7e700 1 -- 172.21.15.43:0/3786629944 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe9c0d0630 msgr2=0x7efe9c0c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:24.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.903+0000 7efea1a7e700 1 --2- 172.21.15.43:0/3786629944 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe9c0d0630 0x7efe9c0c6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.903+0000 7efea1a7e700 1 -- 172.21.15.43:0/3786629944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe9c0cbd40 msgr2=0x7efe9c0b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:24.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.903+0000 7efea1a7e700 1 --2- 172.21.15.43:0/3786629944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe9c0cbd40 0x7efe9c0b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:24.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.903+0000 7efea1a7e700 1 -- 172.21.15.43:0/3786629944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe94009a70 con 0x7efe9c06b540 2023-06-28T20:08:24.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.903+0000 7efea1a7e700 1 --2- 172.21.15.43:0/3786629944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe9c06b540 0x7efe9c0b11f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efe9800c460 tx=0x7efe9800c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:24.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.903+0000 7efe927fc700 1 -- 172.21.15.43:0/3786629944 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98005d00 con 0x7efe9c06b540 2023-06-28T20:08:24.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.903+0000 7efe927fc700 1 -- 172.21.15.43:0/3786629944 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe98005e60 con 0x7efe9c06b540 2023-06-28T20:08:24.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.903+0000 7efea3ce2700 1 -- 172.21.15.43:0/3786629944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe8c009550 con 0x7efe9c06b540 2023-06-28T20:08:24.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.903+0000 7efe927fc700 1 -- 172.21.15.43:0/3786629944 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe98007310 con 0x7efe9c06b540 2023-06-28T20:08:24.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.903+0000 7efea3ce2700 1 -- 172.21.15.43:0/3786629944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9c0b2070 con 0x7efe9c06b540 2023-06-28T20:08:24.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.904+0000 7efea3ce2700 1 -- 172.21.15.43:0/3786629944 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe9c0728b0 con 0x7efe9c06b540 2023-06-28T20:08:24.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.905+0000 7efe927fc700 1 -- 172.21.15.43:0/3786629944 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efe98007470 con 0x7efe9c06b540 2023-06-28T20:08:24.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.905+0000 7efe927fc700 1 --2- 172.21.15.43:0/3786629944 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efe8006ace0 0x7efe8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:24.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.905+0000 7efe927fc700 1 -- 172.21.15.43:0/3786629944 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efe98088780 con 0x7efe9c06b540 2023-06-28T20:08:24.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.905+0000 7efea127d700 1 --2- 172.21.15.43:0/3786629944 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efe8006ace0 0x7efe8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:24.921 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.906+0000 7efea127d700 1 --2- 172.21.15.43:0/3786629944 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efe8006ace0 0x7efe8006d190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efe8c000c00 tx=0x7efe8c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:24.922 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:24.910+0000 7efe927fc700 1 -- 172.21.15.43:0/3786629944 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efe98053830 con 0x7efe9c06b540 2023-06-28T20:08:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.102+0000 7efea3ce2700 1 -- 172.21.15.43:0/3786629944 --> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efe9c0cc5e0 con 0x7efe8006ace0 2023-06-28T20:08:25.115 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.111+0000 7efe927fc700 1 -- 172.21.15.43:0/3786629944 <== mgr.4101 v2:172.21.15.43:6832/406144153 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311484 (secure 0 0 0) 0x7efe8c011b80 con 0x7efe8006ace0 2023-06-28T20:08:25.118 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:25.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.114+0000 7efea3ce2700 1 -- 172.21.15.43:0/3786629944 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efe8006ace0 msgr2=0x7efe8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:25.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.114+0000 7efea3ce2700 1 --2- 172.21.15.43:0/3786629944 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efe8006ace0 0x7efe8006d190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efe8c000c00 tx=0x7efe8c008030 comp rx=0 tx=0).stop 2023-06-28T20:08:25.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.114+0000 7efea3ce2700 1 -- 172.21.15.43:0/3786629944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe9c06b540 msgr2=0x7efe9c0b11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:25.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.114+0000 7efea3ce2700 1 --2- 172.21.15.43:0/3786629944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe9c06b540 0x7efe9c0b11f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efe9800c460 tx=0x7efe9800c770 comp rx=0 tx=0).stop 2023-06-28T20:08:25.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.114+0000 7efea3ce2700 1 -- 172.21.15.43:0/3786629944 shutdown_connections 2023-06-28T20:08:25.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.114+0000 7efea3ce2700 1 --2- 172.21.15.43:0/3786629944 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7efe8006ace0 0x7efe8006d190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.115+0000 7efea3ce2700 1 --2- 172.21.15.43:0/3786629944 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7efe9c0d0630 0x7efe9c0c6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.115+0000 7efea3ce2700 1 --2- 172.21.15.43:0/3786629944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe9c06b540 0x7efe9c0b11f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.115+0000 7efea3ce2700 1 --2- 172.21.15.43:0/3786629944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe9c0cbd40 0x7efe9c0b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.115+0000 7efea3ce2700 1 -- 172.21.15.43:0/3786629944 >> 172.21.15.43:0/3786629944 conn(0x7efe9c0664b0 msgr2=0x7efe9c0add10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:25.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.115+0000 7efea3ce2700 1 -- 172.21.15.43:0/3786629944 shutdown_connections 2023-06-28T20:08:25.122 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.115+0000 7efea3ce2700 1 -- 172.21.15.43:0/3786629944 wait complete. 2023-06-28T20:08:25.122 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-06-28T20:08:25.131 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2023-06-28T20:08:24.003794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":163,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219940,"kb_used_data":4324,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754780,"statfs":{"total":773094113280,"available":772868894720,"internally_reserved":0,"allocated":4427776,"data_stored":2504291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.362094"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109380+0000","last_change":"2023-06-28T20:08:20.679268+0000","last_active":"2023-06-28T20:08:22.109380+0000","last_peered":"2023-06-28T20:08:22.109380+0000","last_clean":"2023-06-28T20:08:22.109380+0000","last_became_active":"2023-06-28T20:08:20.678933+0000","last_became_peered":"2023-06-28T20:08:20.678933+0000","last_unstale":"2023-06-28T20:08:22.109380+0000","last_undegraded":"2023-06-28T20:08:22.109380+0000","last_fullsized":"2023-06-28T20:08:22.109380+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:01:15.843032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108652+0000","last_change":"2023-06-28T20:08:20.685759+0000","last_active":"2023-06-28T20:08:22.108652+0000","last_peered":"2023-06-28T20:08:22.108652+0000","last_clean":"2023-06-28T20:08:22.108652+0000","last_became_active":"2023-06-28T20:08:20.685463+0000","last_became_peered":"2023-06-28T20:08:20.685463+0000","last_unstale":"2023-06-28T20:08:22.108652+0000","last_undegraded":"2023-06-28T20:08:22.108652+0000","last_fullsized":"2023-06-28T20:08:22.108652+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:09:08.348528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108848+0000","last_change":"2023-06-28T20:08:20.680151+0000","last_active":"2023-06-28T20:08:22.108848+0000","last_peered":"2023-06-28T20:08:22.108848+0000","last_clean":"2023-06-28T20:08:22.108848+0000","last_became_active":"2023-06-28T20:08:20.679855+0000","last_became_peered":"2023-06-28T20:08:20.679855+0000","last_unstale":"2023-06-28T20:08:22.108848+0000","last_undegraded":"2023-06-28T20:08:22.108848+0000","last_fullsized":"2023-06-28T20:08:22.108848+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:17:11.674363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.799185+0000","last_change":"2023-06-28T20:08:20.685562+0000","last_active":"2023-06-28T20:08:21.799185+0000","last_peered":"2023-06-28T20:08:21.799185+0000","last_clean":"2023-06-28T20:08:21.799185+0000","last_became_active":"2023-06-28T20:08:20.684942+0000","last_became_peered":"2023-06-28T20:08:20.684942+0000","last_unstale":"2023-06-28T20:08:21.799185+0000","last_undegraded":"2023-06-28T20:08:21.799185+0000","last_fullsized":"2023-06-28T20:08:21.799185+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:06:33.836982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.798759+0000","last_change":"2023-06-28T20:08:20.679584+0000","last_active":"2023-06-28T20:08:21.798759+0000","last_peered":"2023-06-28T20:08:21.798759+0000","last_clean":"2023-06-28T20:08:21.798759+0000","last_became_active":"2023-06-28T20:08:20.679308+0000","last_became_peered":"2023-06-28T20:08:20.679308+0000","last_unstale":"2023-06-28T20:08:21.798759+0000","last_undegraded":"2023-06-28T20:08:21.798759+0000","last_fullsized":"2023-06-28T20:08:21.798759+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:32:28.393764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109307+0000","last_change":"2023-06-28T20:08:20.678092+0000","last_active":"2023-06-28T20:08:22.109307+0000","last_peered":"2023-06-28T20:08:22.109307+0000","last_clean":"2023-06-28T20:08:22.109307+0000","last_became_active":"2023-06-28T20:08:20.677783+0000","last_became_peered":"2023-06-28T20:08:20.677783+0000","last_unstale":"2023-06-28T20:08:22.109307+0000","last_undegraded":"2023-06-28T20:08:22.109307+0000","last_fullsized":"2023-06-28T20:08:22.109307+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:52:01.183214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801212+0000","last_change":"2023-06-28T20:08:20.683460+0000","last_active":"2023-06-28T20:08:21.801212+0000","last_peered":"2023-06-28T20:08:21.801212+0000","last_clean":"2023-06-28T20:08:21.801212+0000","last_became_active":"2023-06-28T20:08:20.682294+0000","last_became_peered":"2023-06-28T20:08:20.682294+0000","last_unstale":"2023-06-28T20:08:21.801212+0000","last_undegraded":"2023-06-28T20:08:21.801212+0000","last_fullsized":"2023-06-28T20:08:21.801212+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:47:32.289655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108529+0000","last_change":"2023-06-28T20:08:20.680634+0000","last_active":"2023-06-28T20:08:22.108529+0000","last_peered":"2023-06-28T20:08:22.108529+0000","last_clean":"2023-06-28T20:08:22.108529+0000","last_became_active":"2023-06-28T20:08:20.679711+0000","last_became_peered":"2023-06-28T20:08:20.679711+0000","last_unstale":"2023-06-28T20:08:22.108529+0000","last_undegraded":"2023-06-28T20:08:22.108529+0000","last_fullsized":"2023-06-28T20:08:22.108529+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:09:34.429733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108391+0000","last_change":"2023-06-28T20:08:20.687504+0000","last_active":"2023-06-28T20:08:22.108391+0000","last_peered":"2023-06-28T20:08:22.108391+0000","last_clean":"2023-06-28T20:08:22.108391+0000","last_became_active":"2023-06-28T20:08:20.686571+0000","last_became_peered":"2023-06-28T20:08:20.686571+0000","last_unstale":"2023-06-28T20:08:22.108391+0000","last_undegraded":"2023-06-28T20:08:22.108391+0000","last_fullsized":"2023-06-28T20:08:22.108391+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:59:13.572447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801525+0000","last_change":"2023-06-28T20:08:20.682066+0000","last_active":"2023-06-28T20:08:21.801525+0000","last_peered":"2023-06-28T20:08:21.801525+0000","last_clean":"2023-06-28T20:08:21.801525+0000","last_became_active":"2023-06-28T20:08:20.681764+0000","last_became_peered":"2023-06-28T20:08:20.681764+0000","last_unstale":"2023-06-28T20:08:21.801525+0000","last_undegraded":"2023-06-28T20:08:21.801525+0000","last_fullsized":"2023-06-28T20:08:21.801525+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:59:28.529049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801518+0000","last_change":"2023-06-28T20:08:20.678239+0000","last_active":"2023-06-28T20:08:21.801518+0000","last_peered":"2023-06-28T20:08:21.801518+0000","last_clean":"2023-06-28T20:08:21.801518+0000","last_became_active":"2023-06-28T20:08:20.677233+0000","last_became_peered":"2023-06-28T20:08:20.677233+0000","last_unstale":"2023-06-28T20:08:21.801518+0000","last_undegraded":"2023-06-28T20:08:21.801518+0000","last_fullsized":"2023-06-28T20:08:21.801518+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:28:20.539013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108313+0000","last_change":"2023-06-28T20:08:20.686755+0000","last_active":"2023-06-28T20:08:22.108313+0000","last_peered":"2023-06-28T20:08:22.108313+0000","last_clean":"2023-06-28T20:08:22.108313+0000","last_became_active":"2023-06-28T20:08:20.686386+0000","last_became_peered":"2023-06-28T20:08:20.686386+0000","last_unstale":"2023-06-28T20:08:22.108313+0000","last_undegraded":"2023-06-28T20:08:22.108313+0000","last_fullsized":"2023-06-28T20:08:22.108313+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:05:46.974291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801475+0000","last_change":"2023-06-28T20:08:20.683528+0000","last_active":"2023-06-28T20:08:21.801475+0000","last_peered":"2023-06-28T20:08:21.801475+0000","last_clean":"2023-06-28T20:08:21.801475+0000","last_became_active":"2023-06-28T20:08:20.681838+0000","last_became_peered":"2023-06-28T20:08:20.681838+0000","last_unstale":"2023-06-28T20:08:21.801475+0000","last_undegraded":"2023-06-28T20:08:21.801475+0000","last_fullsized":"2023-06-28T20:08:21.801475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:21:39.943734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659709+0000","last_change":"2023-06-28T20:08:20.666405+0000","last_active":"2023-06-28T20:08:21.659709+0000","last_peered":"2023-06-28T20:08:21.659709+0000","last_clean":"2023-06-28T20:08:21.659709+0000","last_became_active":"2023-06-28T20:08:20.666130+0000","last_became_peered":"2023-06-28T20:08:20.666130+0000","last_unstale":"2023-06-28T20:08:21.659709+0000","last_undegraded":"2023-06-28T20:08:21.659709+0000","last_fullsized":"2023-06-28T20:08:21.659709+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:32:32.975208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801071+0000","last_change":"2023-06-28T20:08:20.687883+0000","last_active":"2023-06-28T20:08:21.801071+0000","last_peered":"2023-06-28T20:08:21.801071+0000","last_clean":"2023-06-28T20:08:21.801071+0000","last_became_active":"2023-06-28T20:08:20.687583+0000","last_became_peered":"2023-06-28T20:08:20.687583+0000","last_unstale":"2023-06-28T20:08:21.801071+0000","last_undegraded":"2023-06-28T20:08:21.801071+0000","last_fullsized":"2023-06-28T20:08:21.801071+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:52:36.907231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.658620+0000","last_change":"2023-06-28T20:08:20.678870+0000","last_active":"2023-06-28T20:08:21.658620+0000","last_peered":"2023-06-28T20:08:21.658620+0000","last_clean":"2023-06-28T20:08:21.658620+0000","last_became_active":"2023-06-28T20:08:20.678608+0000","last_became_peered":"2023-06-28T20:08:20.678608+0000","last_unstale":"2023-06-28T20:08:21.658620+0000","last_undegraded":"2023-06-28T20:08:21.658620+0000","last_fullsized":"2023-06-28T20:08:21.658620+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:28:20.539013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660565+0000","last_change":"2023-06-28T20:08:20.679525+0000","last_active":"2023-06-28T20:08:21.660565+0000","last_peered":"2023-06-28T20:08:21.660565+0000","last_clean":"2023-06-28T20:08:21.660565+0000","last_became_active":"2023-06-28T20:08:20.679222+0000","last_became_peered":"2023-06-28T20:08:20.679222+0000","last_unstale":"2023-06-28T20:08:21.660565+0000","last_undegraded":"2023-06-28T20:08:21.660565+0000","last_fullsized":"2023-06-28T20:08:21.660565+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:12:22.784860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660002+0000","last_change":"2023-06-28T20:08:20.667998+0000","last_active":"2023-06-28T20:08:21.660002+0000","last_peered":"2023-06-28T20:08:21.660002+0000","last_clean":"2023-06-28T20:08:21.660002+0000","last_became_active":"2023-06-28T20:08:20.665959+0000","last_became_peered":"2023-06-28T20:08:20.665959+0000","last_unstale":"2023-06-28T20:08:21.660002+0000","last_undegraded":"2023-06-28T20:08:21.660002+0000","last_fullsized":"2023-06-28T20:08:21.660002+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:15:54.670164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.798909+0000","last_change":"2023-06-28T20:08:20.687229+0000","last_active":"2023-06-28T20:08:21.798909+0000","last_peered":"2023-06-28T20:08:21.798909+0000","last_clean":"2023-06-28T20:08:21.798909+0000","last_became_active":"2023-06-28T20:08:20.686917+0000","last_became_peered":"2023-06-28T20:08:20.686917+0000","last_unstale":"2023-06-28T20:08:21.798909+0000","last_undegraded":"2023-06-28T20:08:21.798909+0000","last_fullsized":"2023-06-28T20:08:21.798909+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:01:55.475356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660191+0000","last_change":"2023-06-28T20:08:20.666513+0000","last_active":"2023-06-28T20:08:21.660191+0000","last_peered":"2023-06-28T20:08:21.660191+0000","last_clean":"2023-06-28T20:08:21.660191+0000","last_became_active":"2023-06-28T20:08:20.666222+0000","last_became_peered":"2023-06-28T20:08:20.666222+0000","last_unstale":"2023-06-28T20:08:21.660191+0000","last_undegraded":"2023-06-28T20:08:21.660191+0000","last_fullsized":"2023-06-28T20:08:21.660191+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:00:06.813246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801519+0000","last_change":"2023-06-28T20:08:20.679222+0000","last_active":"2023-06-28T20:08:21.801519+0000","last_peered":"2023-06-28T20:08:21.801519+0000","last_clean":"2023-06-28T20:08:21.801519+0000","last_became_active":"2023-06-28T20:08:20.678940+0000","last_became_peered":"2023-06-28T20:08:20.678940+0000","last_unstale":"2023-06-28T20:08:21.801519+0000","last_undegraded":"2023-06-28T20:08:21.801519+0000","last_fullsized":"2023-06-28T20:08:21.801519+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:00:06.813246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659165+0000","last_change":"2023-06-28T20:08:20.684209+0000","last_active":"2023-06-28T20:08:21.659165+0000","last_peered":"2023-06-28T20:08:21.659165+0000","last_clean":"2023-06-28T20:08:21.659165+0000","last_became_active":"2023-06-28T20:08:20.683888+0000","last_became_peered":"2023-06-28T20:08:20.683888+0000","last_unstale":"2023-06-28T20:08:21.659165+0000","last_undegraded":"2023-06-28T20:08:21.659165+0000","last_fullsized":"2023-06-28T20:08:21.659165+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:12:02.588448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659187+0000","last_change":"2023-06-28T20:08:20.685467+0000","last_active":"2023-06-28T20:08:21.659187+0000","last_peered":"2023-06-28T20:08:21.659187+0000","last_clean":"2023-06-28T20:08:21.659187+0000","last_became_active":"2023-06-28T20:08:20.685099+0000","last_became_peered":"2023-06-28T20:08:20.685099+0000","last_unstale":"2023-06-28T20:08:21.659187+0000","last_undegraded":"2023-06-28T20:08:21.659187+0000","last_fullsized":"2023-06-28T20:08:21.659187+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:42:21.632954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.798926+0000","last_change":"2023-06-28T20:08:20.685718+0000","last_active":"2023-06-28T20:08:21.798926+0000","last_peered":"2023-06-28T20:08:21.798926+0000","last_clean":"2023-06-28T20:08:21.798926+0000","last_became_active":"2023-06-28T20:08:20.684773+0000","last_became_peered":"2023-06-28T20:08:20.684773+0000","last_unstale":"2023-06-28T20:08:21.798926+0000","last_undegraded":"2023-06-28T20:08:21.798926+0000","last_fullsized":"2023-06-28T20:08:21.798926+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:56:30.427117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660102+0000","last_change":"2023-06-28T20:08:20.676377+0000","last_active":"2023-06-28T20:08:21.660102+0000","last_peered":"2023-06-28T20:08:21.660102+0000","last_clean":"2023-06-28T20:08:21.660102+0000","last_became_active":"2023-06-28T20:08:20.675433+0000","last_became_peered":"2023-06-28T20:08:20.675433+0000","last_unstale":"2023-06-28T20:08:21.660102+0000","last_undegraded":"2023-06-28T20:08:21.660102+0000","last_fullsized":"2023-06-28T20:08:21.660102+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:19:46.143213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109555+0000","last_change":"2023-06-28T20:08:20.679204+0000","last_active":"2023-06-28T20:08:22.109555+0000","last_peered":"2023-06-28T20:08:22.109555+0000","last_clean":"2023-06-28T20:08:22.109555+0000","last_became_active":"2023-06-28T20:08:20.678942+0000","last_became_peered":"2023-06-28T20:08:20.678942+0000","last_unstale":"2023-06-28T20:08:22.109555+0000","last_undegraded":"2023-06-28T20:08:22.109555+0000","last_fullsized":"2023-06-28T20:08:22.109555+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:17:11.674363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660081+0000","last_change":"2023-06-28T20:08:20.687837+0000","last_active":"2023-06-28T20:08:21.660081+0000","last_peered":"2023-06-28T20:08:21.660081+0000","last_clean":"2023-06-28T20:08:21.660081+0000","last_became_active":"2023-06-28T20:08:20.687372+0000","last_became_peered":"2023-06-28T20:08:20.687372+0000","last_unstale":"2023-06-28T20:08:21.660081+0000","last_undegraded":"2023-06-28T20:08:21.660081+0000","last_fullsized":"2023-06-28T20:08:21.660081+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:05:30.000982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801377+0000","last_change":"2023-06-28T20:08:20.689285+0000","last_active":"2023-06-28T20:08:21.801377+0000","last_peered":"2023-06-28T20:08:21.801377+0000","last_clean":"2023-06-28T20:08:21.801377+0000","last_became_active":"2023-06-28T20:08:20.689030+0000","last_became_peered":"2023-06-28T20:08:20.689030+0000","last_unstale":"2023-06-28T20:08:21.801377+0000","last_undegraded":"2023-06-28T20:08:21.801377+0000","last_fullsized":"2023-06-28T20:08:21.801377+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:05:30.000982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659542+0000","last_change":"2023-06-28T20:08:20.667476+0000","last_active":"2023-06-28T20:08:21.659542+0000","last_peered":"2023-06-28T20:08:21.659542+0000","last_clean":"2023-06-28T20:08:21.659542+0000","last_became_active":"2023-06-28T20:08:20.667132+0000","last_became_peered":"2023-06-28T20:08:20.667132+0000","last_unstale":"2023-06-28T20:08:21.659542+0000","last_undegraded":"2023-06-28T20:08:21.659542+0000","last_fullsized":"2023-06-28T20:08:21.659542+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:21:39.943734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660362+0000","last_change":"2023-06-28T20:08:20.667291+0000","last_active":"2023-06-28T20:08:21.660362+0000","last_peered":"2023-06-28T20:08:21.660362+0000","last_clean":"2023-06-28T20:08:21.660362+0000","last_became_active":"2023-06-28T20:08:20.666522+0000","last_became_peered":"2023-06-28T20:08:20.666522+0000","last_unstale":"2023-06-28T20:08:21.660362+0000","last_undegraded":"2023-06-28T20:08:21.660362+0000","last_fullsized":"2023-06-28T20:08:21.660362+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:57:33.386641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.658267+0000","last_change":"2023-06-28T20:08:20.680130+0000","last_active":"2023-06-28T20:08:21.658267+0000","last_peered":"2023-06-28T20:08:21.658267+0000","last_clean":"2023-06-28T20:08:21.658267+0000","last_became_active":"2023-06-28T20:08:20.679865+0000","last_became_peered":"2023-06-28T20:08:20.679865+0000","last_unstale":"2023-06-28T20:08:21.658267+0000","last_undegraded":"2023-06-28T20:08:21.658267+0000","last_fullsized":"2023-06-28T20:08:21.658267+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:59:47.246466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108041+0000","last_change":"2023-06-28T20:08:20.685641+0000","last_active":"2023-06-28T20:08:22.108041+0000","last_peered":"2023-06-28T20:08:22.108041+0000","last_clean":"2023-06-28T20:08:22.108041+0000","last_became_active":"2023-06-28T20:08:20.685264+0000","last_became_peered":"2023-06-28T20:08:20.685264+0000","last_unstale":"2023-06-28T20:08:22.108041+0000","last_undegraded":"2023-06-28T20:08:22.108041+0000","last_fullsized":"2023-06-28T20:08:22.108041+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:58:25.349192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.658855+0000","last_change":"2023-06-28T20:08:20.682793+0000","last_active":"2023-06-28T20:08:21.658855+0000","last_peered":"2023-06-28T20:08:21.658855+0000","last_clean":"2023-06-28T20:08:21.658855+0000","last_became_active":"2023-06-28T20:08:20.682482+0000","last_became_peered":"2023-06-28T20:08:20.682482+0000","last_unstale":"2023-06-28T20:08:21.658855+0000","last_undegraded":"2023-06-28T20:08:21.658855+0000","last_fullsized":"2023-06-28T20:08:21.658855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:58:15.735553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"34'4","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660257+0000","last_change":"2023-06-28T20:08:19.655460+0000","last_active":"2023-06-28T20:08:21.660257+0000","last_peered":"2023-06-28T20:08:21.660257+0000","last_clean":"2023-06-28T20:08:21.660257+0000","last_became_active":"2023-06-28T20:08:19.654505+0000","last_became_peered":"2023-06-28T20:08:19.654505+0000","last_unstale":"2023-06-28T20:08:21.660257+0000","last_undegraded":"2023-06-28T20:08:21.660257+0000","last_fullsized":"2023-06-28T20:08:21.660257+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:59:27.524440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109721+0000","last_change":"2023-06-28T20:08:20.680757+0000","last_active":"2023-06-28T20:08:22.109721+0000","last_peered":"2023-06-28T20:08:22.109721+0000","last_clean":"2023-06-28T20:08:22.109721+0000","last_became_active":"2023-06-28T20:08:20.680461+0000","last_became_peered":"2023-06-28T20:08:20.680461+0000","last_unstale":"2023-06-28T20:08:22.109721+0000","last_undegraded":"2023-06-28T20:08:22.109721+0000","last_fullsized":"2023-06-28T20:08:22.109721+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:15:25.870011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108368+0000","last_change":"2023-06-28T20:08:19.661131+0000","last_active":"2023-06-28T20:08:22.108368+0000","last_peered":"2023-06-28T20:08:22.108368+0000","last_clean":"2023-06-28T20:08:22.108368+0000","last_became_active":"2023-06-28T20:08:19.660874+0000","last_became_peered":"2023-06-28T20:08:19.660874+0000","last_unstale":"2023-06-28T20:08:22.108368+0000","last_undegraded":"2023-06-28T20:08:22.108368+0000","last_fullsized":"2023-06-28T20:08:22.108368+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:52:00.178605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659785+0000","last_change":"2023-06-28T20:08:20.665348+0000","last_active":"2023-06-28T20:08:21.659785+0000","last_peered":"2023-06-28T20:08:21.659785+0000","last_clean":"2023-06-28T20:08:21.659785+0000","last_became_active":"2023-06-28T20:08:20.664971+0000","last_became_peered":"2023-06-28T20:08:20.664971+0000","last_unstale":"2023-06-28T20:08:21.659785+0000","last_undegraded":"2023-06-28T20:08:21.659785+0000","last_fullsized":"2023-06-28T20:08:21.659785+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:20:18.739296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109782+0000","last_change":"2023-06-28T20:08:19.662313+0000","last_active":"2023-06-28T20:08:22.109782+0000","last_peered":"2023-06-28T20:08:22.109782+0000","last_clean":"2023-06-28T20:08:22.109782+0000","last_became_active":"2023-06-28T20:08:19.661987+0000","last_became_peered":"2023-06-28T20:08:19.661987+0000","last_unstale":"2023-06-28T20:08:22.109782+0000","last_undegraded":"2023-06-28T20:08:22.109782+0000","last_fullsized":"2023-06-28T20:08:22.109782+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:35:24.723033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109079+0000","last_change":"2023-06-28T20:08:20.675769+0000","last_active":"2023-06-28T20:08:22.109079+0000","last_peered":"2023-06-28T20:08:22.109079+0000","last_clean":"2023-06-28T20:08:22.109079+0000","last_became_active":"2023-06-28T20:08:20.675425+0000","last_became_peered":"2023-06-28T20:08:20.675425+0000","last_unstale":"2023-06-28T20:08:22.109079+0000","last_undegraded":"2023-06-28T20:08:22.109079+0000","last_fullsized":"2023-06-28T20:08:22.109079+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:57:32.895917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659585+0000","last_change":"2023-06-28T20:08:19.667027+0000","last_active":"2023-06-28T20:08:21.659585+0000","last_peered":"2023-06-28T20:08:21.659585+0000","last_clean":"2023-06-28T20:08:21.659585+0000","last_became_active":"2023-06-28T20:08:19.666747+0000","last_became_peered":"2023-06-28T20:08:19.666747+0000","last_unstale":"2023-06-28T20:08:21.659585+0000","last_undegraded":"2023-06-28T20:08:21.659585+0000","last_fullsized":"2023-06-28T20:08:21.659585+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:28:50.129574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108175+0000","last_change":"2023-06-28T20:08:20.677333+0000","last_active":"2023-06-28T20:08:22.108175+0000","last_peered":"2023-06-28T20:08:22.108175+0000","last_clean":"2023-06-28T20:08:22.108175+0000","last_became_active":"2023-06-28T20:08:20.677065+0000","last_became_peered":"2023-06-28T20:08:20.677065+0000","last_unstale":"2023-06-28T20:08:22.108175+0000","last_undegraded":"2023-06-28T20:08:22.108175+0000","last_fullsized":"2023-06-28T20:08:22.108175+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:11:02.786663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801292+0000","last_change":"2023-06-28T20:08:19.665791+0000","last_active":"2023-06-28T20:08:21.801292+0000","last_peered":"2023-06-28T20:08:21.801292+0000","last_clean":"2023-06-28T20:08:21.801292+0000","last_became_active":"2023-06-28T20:08:19.665360+0000","last_became_peered":"2023-06-28T20:08:19.665360+0000","last_unstale":"2023-06-28T20:08:21.801292+0000","last_undegraded":"2023-06-28T20:08:21.801292+0000","last_fullsized":"2023-06-28T20:08:21.801292+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:15:53.665554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801077+0000","last_change":"2023-06-28T20:08:20.684309+0000","last_active":"2023-06-28T20:08:21.801077+0000","last_peered":"2023-06-28T20:08:21.801077+0000","last_clean":"2023-06-28T20:08:21.801077+0000","last_became_active":"2023-06-28T20:08:20.684033+0000","last_became_peered":"2023-06-28T20:08:20.684033+0000","last_unstale":"2023-06-28T20:08:21.801077+0000","last_undegraded":"2023-06-28T20:08:21.801077+0000","last_fullsized":"2023-06-28T20:08:21.801077+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:35:25.727643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.658559+0000","last_change":"2023-06-28T20:08:19.663213+0000","last_active":"2023-06-28T20:08:21.658559+0000","last_peered":"2023-06-28T20:08:21.658559+0000","last_clean":"2023-06-28T20:08:21.658559+0000","last_became_active":"2023-06-28T20:08:19.662945+0000","last_became_peered":"2023-06-28T20:08:19.662945+0000","last_unstale":"2023-06-28T20:08:21.658559+0000","last_undegraded":"2023-06-28T20:08:21.658559+0000","last_fullsized":"2023-06-28T20:08:21.658559+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:00:05.808637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659321+0000","last_change":"2023-06-28T20:08:20.680144+0000","last_active":"2023-06-28T20:08:21.659321+0000","last_peered":"2023-06-28T20:08:21.659321+0000","last_clean":"2023-06-28T20:08:21.659321+0000","last_became_active":"2023-06-28T20:08:20.679612+0000","last_became_peered":"2023-06-28T20:08:20.679612+0000","last_unstale":"2023-06-28T20:08:21.659321+0000","last_undegraded":"2023-06-28T20:08:21.659321+0000","last_fullsized":"2023-06-28T20:08:21.659321+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:53:20.326887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109273+0000","last_change":"2023-06-28T20:08:19.664997+0000","last_active":"2023-06-28T20:08:22.109273+0000","last_peered":"2023-06-28T20:08:22.109273+0000","last_clean":"2023-06-28T20:08:22.109273+0000","last_became_active":"2023-06-28T20:08:19.664694+0000","last_became_peered":"2023-06-28T20:08:19.664694+0000","last_unstale":"2023-06-28T20:08:22.109273+0000","last_undegraded":"2023-06-28T20:08:22.109273+0000","last_fullsized":"2023-06-28T20:08:22.109273+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:05:45.969682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109030+0000","last_change":"2023-06-28T20:08:20.670512+0000","last_active":"2023-06-28T20:08:22.109030+0000","last_peered":"2023-06-28T20:08:22.109030+0000","last_clean":"2023-06-28T20:08:22.109030+0000","last_became_active":"2023-06-28T20:08:20.669897+0000","last_became_peered":"2023-06-28T20:08:20.669897+0000","last_unstale":"2023-06-28T20:08:22.109030+0000","last_undegraded":"2023-06-28T20:08:22.109030+0000","last_fullsized":"2023-06-28T20:08:22.109030+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:54:06.279406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659345+0000","last_change":"2023-06-28T20:08:19.652654+0000","last_active":"2023-06-28T20:08:21.659345+0000","last_peered":"2023-06-28T20:08:21.659345+0000","last_clean":"2023-06-28T20:08:21.659345+0000","last_became_active":"2023-06-28T20:08:19.652390+0000","last_became_peered":"2023-06-28T20:08:19.652390+0000","last_unstale":"2023-06-28T20:08:21.659345+0000","last_undegraded":"2023-06-28T20:08:21.659345+0000","last_fullsized":"2023-06-28T20:08:21.659345+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:03:20.512184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.798969+0000","last_change":"2023-06-28T20:08:20.685963+0000","last_active":"2023-06-28T20:08:21.798969+0000","last_peered":"2023-06-28T20:08:21.798969+0000","last_clean":"2023-06-28T20:08:21.798969+0000","last_became_active":"2023-06-28T20:08:20.685676+0000","last_became_peered":"2023-06-28T20:08:20.685676+0000","last_unstale":"2023-06-28T20:08:21.798969+0000","last_undegraded":"2023-06-28T20:08:21.798969+0000","last_fullsized":"2023-06-28T20:08:21.798969+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:58:50.702496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109250+0000","last_change":"2023-06-28T20:08:19.666008+0000","last_active":"2023-06-28T20:08:22.109250+0000","last_peered":"2023-06-28T20:08:22.109250+0000","last_clean":"2023-06-28T20:08:22.109250+0000","last_became_active":"2023-06-28T20:08:19.665730+0000","last_became_peered":"2023-06-28T20:08:19.665730+0000","last_unstale":"2023-06-28T20:08:22.109250+0000","last_undegraded":"2023-06-28T20:08:22.109250+0000","last_fullsized":"2023-06-28T20:08:22.109250+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:04:49.884359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660250+0000","last_change":"2023-06-28T20:08:20.667701+0000","last_active":"2023-06-28T20:08:21.660250+0000","last_peered":"2023-06-28T20:08:21.660250+0000","last_clean":"2023-06-28T20:08:21.660250+0000","last_became_active":"2023-06-28T20:08:20.667195+0000","last_became_peered":"2023-06-28T20:08:20.667195+0000","last_unstale":"2023-06-28T20:08:21.660250+0000","last_undegraded":"2023-06-28T20:08:21.660250+0000","last_fullsized":"2023-06-28T20:08:21.660250+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:01:31.303207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801689+0000","last_change":"2023-06-28T20:08:19.667830+0000","last_active":"2023-06-28T20:08:21.801689+0000","last_peered":"2023-06-28T20:08:21.801689+0000","last_clean":"2023-06-28T20:08:21.801689+0000","last_became_active":"2023-06-28T20:08:19.667590+0000","last_became_peered":"2023-06-28T20:08:19.667590+0000","last_unstale":"2023-06-28T20:08:21.801689+0000","last_undegraded":"2023-06-28T20:08:21.801689+0000","last_fullsized":"2023-06-28T20:08:21.801689+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:58:14.730943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109735+0000","last_change":"2023-06-28T20:08:20.676603+0000","last_active":"2023-06-28T20:08:22.109735+0000","last_peered":"2023-06-28T20:08:22.109735+0000","last_clean":"2023-06-28T20:08:22.109735+0000","last_became_active":"2023-06-28T20:08:20.676293+0000","last_became_peered":"2023-06-28T20:08:20.676293+0000","last_unstale":"2023-06-28T20:08:22.109735+0000","last_undegraded":"2023-06-28T20:08:22.109735+0000","last_fullsized":"2023-06-28T20:08:22.109735+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:04:56.887142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"34'3","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801684+0000","last_change":"2023-06-28T20:08:19.666930+0000","last_active":"2023-06-28T20:08:21.801684+0000","last_peered":"2023-06-28T20:08:21.801684+0000","last_clean":"2023-06-28T20:08:21.801684+0000","last_became_active":"2023-06-28T20:08:19.666639+0000","last_became_peered":"2023-06-28T20:08:19.666639+0000","last_unstale":"2023-06-28T20:08:21.801684+0000","last_undegraded":"2023-06-28T20:08:21.801684+0000","last_fullsized":"2023-06-28T20:08:21.801684+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:57:32.382031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659780+0000","last_change":"2023-06-28T20:08:20.685705+0000","last_active":"2023-06-28T20:08:21.659780+0000","last_peered":"2023-06-28T20:08:21.659780+0000","last_clean":"2023-06-28T20:08:21.659780+0000","last_became_active":"2023-06-28T20:08:20.685410+0000","last_became_peered":"2023-06-28T20:08:20.685410+0000","last_unstale":"2023-06-28T20:08:21.659780+0000","last_undegraded":"2023-06-28T20:08:21.659780+0000","last_fullsized":"2023-06-28T20:08:21.659780+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:16:41.819174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"34'6","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108711+0000","last_change":"2023-06-28T20:08:19.667034+0000","last_active":"2023-06-28T20:08:22.108711+0000","last_peered":"2023-06-28T20:08:22.108711+0000","last_clean":"2023-06-28T20:08:22.108711+0000","last_became_active":"2023-06-28T20:08:19.666621+0000","last_became_peered":"2023-06-28T20:08:19.666621+0000","last_unstale":"2023-06-28T20:08:22.108711+0000","last_undegraded":"2023-06-28T20:08:22.108711+0000","last_fullsized":"2023-06-28T20:08:22.108711+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:13:48.941368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109853+0000","last_change":"2023-06-28T20:08:20.681551+0000","last_active":"2023-06-28T20:08:22.109853+0000","last_peered":"2023-06-28T20:08:22.109853+0000","last_clean":"2023-06-28T20:08:22.109853+0000","last_became_active":"2023-06-28T20:08:20.681224+0000","last_became_peered":"2023-06-28T20:08:20.681224+0000","last_unstale":"2023-06-28T20:08:22.109853+0000","last_undegraded":"2023-06-28T20:08:22.109853+0000","last_fullsized":"2023-06-28T20:08:22.109853+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:52:36.907231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109088+0000","last_change":"2023-06-28T20:08:19.664339+0000","last_active":"2023-06-28T20:08:22.109088+0000","last_peered":"2023-06-28T20:08:22.109088+0000","last_clean":"2023-06-28T20:08:22.109088+0000","last_became_active":"2023-06-28T20:08:19.664043+0000","last_became_peered":"2023-06-28T20:08:19.664043+0000","last_unstale":"2023-06-28T20:08:22.109088+0000","last_undegraded":"2023-06-28T20:08:22.109088+0000","last_fullsized":"2023-06-28T20:08:22.109088+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:03:23.231723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108943+0000","last_change":"2023-06-28T20:08:20.675759+0000","last_active":"2023-06-28T20:08:22.108943+0000","last_peered":"2023-06-28T20:08:22.108943+0000","last_clean":"2023-06-28T20:08:22.108943+0000","last_became_active":"2023-06-28T20:08:20.675395+0000","last_became_peered":"2023-06-28T20:08:20.675395+0000","last_unstale":"2023-06-28T20:08:22.108943+0000","last_undegraded":"2023-06-28T20:08:22.108943+0000","last_fullsized":"2023-06-28T20:08:22.108943+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:00:56.177481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109633+0000","last_change":"2023-06-28T20:08:19.667168+0000","last_active":"2023-06-28T20:08:22.109633+0000","last_peered":"2023-06-28T20:08:22.109633+0000","last_clean":"2023-06-28T20:08:22.109633+0000","last_became_active":"2023-06-28T20:08:19.666902+0000","last_became_peered":"2023-06-28T20:08:19.666902+0000","last_unstale":"2023-06-28T20:08:22.109633+0000","last_undegraded":"2023-06-28T20:08:22.109633+0000","last_fullsized":"2023-06-28T20:08:22.109633+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:04:55.882533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.658311+0000","last_change":"2023-06-28T20:08:20.684341+0000","last_active":"2023-06-28T20:08:21.658311+0000","last_peered":"2023-06-28T20:08:21.658311+0000","last_clean":"2023-06-28T20:08:21.658311+0000","last_became_active":"2023-06-28T20:08:20.684028+0000","last_became_peered":"2023-06-28T20:08:20.684028+0000","last_unstale":"2023-06-28T20:08:21.658311+0000","last_undegraded":"2023-06-28T20:08:21.658311+0000","last_fullsized":"2023-06-28T20:08:21.658311+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:15:54.670164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659744+0000","last_change":"2023-06-28T20:08:19.653185+0000","last_active":"2023-06-28T20:08:21.659744+0000","last_peered":"2023-06-28T20:08:21.659744+0000","last_clean":"2023-06-28T20:08:21.659744+0000","last_became_active":"2023-06-28T20:08:19.652574+0000","last_became_peered":"2023-06-28T20:08:19.652574+0000","last_unstale":"2023-06-28T20:08:21.659744+0000","last_undegraded":"2023-06-28T20:08:21.659744+0000","last_fullsized":"2023-06-28T20:08:21.659744+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:59:46.241857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108883+0000","last_change":"2023-06-28T20:08:20.679542+0000","last_active":"2023-06-28T20:08:22.108883+0000","last_peered":"2023-06-28T20:08:22.108883+0000","last_clean":"2023-06-28T20:08:22.108883+0000","last_became_active":"2023-06-28T20:08:20.679243+0000","last_became_peered":"2023-06-28T20:08:20.679243+0000","last_unstale":"2023-06-28T20:08:22.108883+0000","last_undegraded":"2023-06-28T20:08:22.108883+0000","last_fullsized":"2023-06-28T20:08:22.108883+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:39:58.915632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"34'8","reported_seq":21,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659344+0000","last_change":"2023-06-28T20:08:19.663890+0000","last_active":"2023-06-28T20:08:21.659344+0000","last_peered":"2023-06-28T20:08:21.659344+0000","last_clean":"2023-06-28T20:08:21.659344+0000","last_became_active":"2023-06-28T20:08:19.663624+0000","last_became_peered":"2023-06-28T20:08:19.663624+0000","last_unstale":"2023-06-28T20:08:21.659344+0000","last_undegraded":"2023-06-28T20:08:21.659344+0000","last_fullsized":"2023-06-28T20:08:21.659344+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:22:55.905699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108621+0000","last_change":"2023-06-28T20:08:20.670804+0000","last_active":"2023-06-28T20:08:22.108621+0000","last_peered":"2023-06-28T20:08:22.108621+0000","last_clean":"2023-06-28T20:08:22.108621+0000","last_became_active":"2023-06-28T20:08:20.670467+0000","last_became_peered":"2023-06-28T20:08:20.670467+0000","last_unstale":"2023-06-28T20:08:22.108621+0000","last_undegraded":"2023-06-28T20:08:22.108621+0000","last_fullsized":"2023-06-28T20:08:22.108621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:45:08.309241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801739+0000","last_change":"2023-06-28T20:08:19.663175+0000","last_active":"2023-06-28T20:08:21.801739+0000","last_peered":"2023-06-28T20:08:21.801739+0000","last_clean":"2023-06-28T20:08:21.801739+0000","last_became_active":"2023-06-28T20:08:19.662835+0000","last_became_peered":"2023-06-28T20:08:19.662835+0000","last_unstale":"2023-06-28T20:08:21.801739+0000","last_undegraded":"2023-06-28T20:08:21.801739+0000","last_fullsized":"2023-06-28T20:08:21.801739+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:42:20.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801527+0000","last_change":"2023-06-28T20:08:20.684077+0000","last_active":"2023-06-28T20:08:21.801527+0000","last_peered":"2023-06-28T20:08:21.801527+0000","last_clean":"2023-06-28T20:08:21.801527+0000","last_became_active":"2023-06-28T20:08:20.683782+0000","last_became_peered":"2023-06-28T20:08:20.683782+0000","last_unstale":"2023-06-28T20:08:21.801527+0000","last_undegraded":"2023-06-28T20:08:21.801527+0000","last_fullsized":"2023-06-28T20:08:21.801527+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:19:46.143213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659977+0000","last_change":"2023-06-28T20:08:19.664515+0000","last_active":"2023-06-28T20:08:21.659977+0000","last_peered":"2023-06-28T20:08:21.659977+0000","last_clean":"2023-06-28T20:08:21.659977+0000","last_became_active":"2023-06-28T20:08:19.663664+0000","last_became_peered":"2023-06-28T20:08:19.663664+0000","last_unstale":"2023-06-28T20:08:21.659977+0000","last_undegraded":"2023-06-28T20:08:21.659977+0000","last_fullsized":"2023-06-28T20:08:21.659977+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:31:54.171322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659901+0000","last_change":"2023-06-28T20:08:20.667446+0000","last_active":"2023-06-28T20:08:21.659901+0000","last_peered":"2023-06-28T20:08:21.659901+0000","last_clean":"2023-06-28T20:08:21.659901+0000","last_became_active":"2023-06-28T20:08:20.667193+0000","last_became_peered":"2023-06-28T20:08:20.667193+0000","last_unstale":"2023-06-28T20:08:21.659901+0000","last_undegraded":"2023-06-28T20:08:21.659901+0000","last_fullsized":"2023-06-28T20:08:21.659901+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:13:52.178823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.799066+0000","last_change":"2023-06-28T20:08:19.662299+0000","last_active":"2023-06-28T20:08:21.799066+0000","last_peered":"2023-06-28T20:08:21.799066+0000","last_clean":"2023-06-28T20:08:21.799066+0000","last_became_active":"2023-06-28T20:08:19.662002+0000","last_became_peered":"2023-06-28T20:08:19.662002+0000","last_unstale":"2023-06-28T20:08:21.799066+0000","last_undegraded":"2023-06-28T20:08:21.799066+0000","last_fullsized":"2023-06-28T20:08:21.799066+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:16:57.488453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.798999+0000","last_change":"2023-06-28T20:08:20.688515+0000","last_active":"2023-06-28T20:08:21.798999+0000","last_peered":"2023-06-28T20:08:21.798999+0000","last_clean":"2023-06-28T20:08:21.798999+0000","last_became_active":"2023-06-28T20:08:20.688241+0000","last_became_peered":"2023-06-28T20:08:20.688241+0000","last_unstale":"2023-06-28T20:08:21.798999+0000","last_undegraded":"2023-06-28T20:08:21.798999+0000","last_fullsized":"2023-06-28T20:08:21.798999+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:48:31.571168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109592+0000","last_change":"2023-06-28T20:08:19.661197+0000","last_active":"2023-06-28T20:08:22.109592+0000","last_peered":"2023-06-28T20:08:22.109592+0000","last_clean":"2023-06-28T20:08:22.109592+0000","last_became_active":"2023-06-28T20:08:19.660788+0000","last_became_peered":"2023-06-28T20:08:19.660788+0000","last_unstale":"2023-06-28T20:08:22.109592+0000","last_undegraded":"2023-06-28T20:08:22.109592+0000","last_fullsized":"2023-06-28T20:08:22.109592+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:39:24.497671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660440+0000","last_change":"2023-06-28T20:08:20.686973+0000","last_active":"2023-06-28T20:08:21.660440+0000","last_peered":"2023-06-28T20:08:21.660440+0000","last_clean":"2023-06-28T20:08:21.660440+0000","last_became_active":"2023-06-28T20:08:20.686719+0000","last_became_peered":"2023-06-28T20:08:20.686719+0000","last_unstale":"2023-06-28T20:08:21.660440+0000","last_undegraded":"2023-06-28T20:08:21.660440+0000","last_fullsized":"2023-06-28T20:08:21.660440+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:57:33.386641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.798859+0000","last_change":"2023-06-28T20:08:19.653505+0000","last_active":"2023-06-28T20:08:21.798859+0000","last_peered":"2023-06-28T20:08:21.798859+0000","last_clean":"2023-06-28T20:08:21.798859+0000","last_became_active":"2023-06-28T20:08:19.653079+0000","last_became_peered":"2023-06-28T20:08:19.653079+0000","last_unstale":"2023-06-28T20:08:21.798859+0000","last_undegraded":"2023-06-28T20:08:21.798859+0000","last_fullsized":"2023-06-28T20:08:21.798859+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:18:23.057931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109808+0000","last_change":"2023-06-28T20:08:20.675012+0000","last_active":"2023-06-28T20:08:22.109808+0000","last_peered":"2023-06-28T20:08:22.109808+0000","last_clean":"2023-06-28T20:08:22.109808+0000","last_became_active":"2023-06-28T20:08:20.674703+0000","last_became_peered":"2023-06-28T20:08:20.674703+0000","last_unstale":"2023-06-28T20:08:22.109808+0000","last_undegraded":"2023-06-28T20:08:22.109808+0000","last_fullsized":"2023-06-28T20:08:22.109808+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:35:25.727643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.798741+0000","last_change":"2023-06-28T20:08:19.652706+0000","last_active":"2023-06-28T20:08:21.798741+0000","last_peered":"2023-06-28T20:08:21.798741+0000","last_clean":"2023-06-28T20:08:21.798741+0000","last_became_active":"2023-06-28T20:08:19.652359+0000","last_became_peered":"2023-06-28T20:08:19.652359+0000","last_unstale":"2023-06-28T20:08:21.798741+0000","last_undegraded":"2023-06-28T20:08:21.798741+0000","last_fullsized":"2023-06-28T20:08:21.798741+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:18:28.900535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801156+0000","last_change":"2023-06-28T20:08:20.679109+0000","last_active":"2023-06-28T20:08:21.801156+0000","last_peered":"2023-06-28T20:08:21.801156+0000","last_clean":"2023-06-28T20:08:21.801156+0000","last_became_active":"2023-06-28T20:08:20.678833+0000","last_became_peered":"2023-06-28T20:08:20.678833+0000","last_unstale":"2023-06-28T20:08:21.801156+0000","last_undegraded":"2023-06-28T20:08:21.801156+0000","last_fullsized":"2023-06-28T20:08:21.801156+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:25:08.187049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.658489+0000","last_change":"2023-06-28T20:08:19.664718+0000","last_active":"2023-06-28T20:08:21.658489+0000","last_peered":"2023-06-28T20:08:21.658489+0000","last_clean":"2023-06-28T20:08:21.658489+0000","last_became_active":"2023-06-28T20:08:19.664456+0000","last_became_peered":"2023-06-28T20:08:19.664456+0000","last_unstale":"2023-06-28T20:08:21.658489+0000","last_undegraded":"2023-06-28T20:08:21.658489+0000","last_fullsized":"2023-06-28T20:08:21.658489+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:59:27.524440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108945+0000","last_change":"2023-06-28T20:08:20.670715+0000","last_active":"2023-06-28T20:08:22.108945+0000","last_peered":"2023-06-28T20:08:22.108945+0000","last_clean":"2023-06-28T20:08:22.108945+0000","last_became_active":"2023-06-28T20:08:20.670255+0000","last_became_peered":"2023-06-28T20:08:20.670255+0000","last_unstale":"2023-06-28T20:08:22.108945+0000","last_undegraded":"2023-06-28T20:08:22.108945+0000","last_fullsized":"2023-06-28T20:08:22.108945+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:45:29.637108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.798735+0000","last_change":"2023-06-28T20:08:19.659806+0000","last_active":"2023-06-28T20:08:21.798735+0000","last_peered":"2023-06-28T20:08:21.798735+0000","last_clean":"2023-06-28T20:08:21.798735+0000","last_became_active":"2023-06-28T20:08:19.659485+0000","last_became_peered":"2023-06-28T20:08:19.659485+0000","last_unstale":"2023-06-28T20:08:21.798735+0000","last_undegraded":"2023-06-28T20:08:21.798735+0000","last_fullsized":"2023-06-28T20:08:21.798735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:26:26.268929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659905+0000","last_change":"2023-06-28T20:08:20.673389+0000","last_active":"2023-06-28T20:08:21.659905+0000","last_peered":"2023-06-28T20:08:21.659905+0000","last_clean":"2023-06-28T20:08:21.659905+0000","last_became_active":"2023-06-28T20:08:20.672447+0000","last_became_peered":"2023-06-28T20:08:20.672447+0000","last_unstale":"2023-06-28T20:08:21.659905+0000","last_undegraded":"2023-06-28T20:08:21.659905+0000","last_fullsized":"2023-06-28T20:08:21.659905+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:31:45.241155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659003+0000","last_change":"2023-06-28T20:08:19.665324+0000","last_active":"2023-06-28T20:08:21.659003+0000","last_peered":"2023-06-28T20:08:21.659003+0000","last_clean":"2023-06-28T20:08:21.659003+0000","last_became_active":"2023-06-28T20:08:19.664984+0000","last_became_peered":"2023-06-28T20:08:19.664984+0000","last_unstale":"2023-06-28T20:08:21.659003+0000","last_undegraded":"2023-06-28T20:08:21.659003+0000","last_fullsized":"2023-06-28T20:08:21.659003+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:19:03.246307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":37,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659501+0000","last_change":"2023-06-28T20:07:15.815425+0000","last_active":"2023-06-28T20:08:21.659501+0000","last_peered":"2023-06-28T20:08:21.659501+0000","last_clean":"2023-06-28T20:08:21.659501+0000","last_became_active":"2023-06-28T20:07:15.815163+0000","last_became_peered":"2023-06-28T20:07:15.815163+0000","last_unstale":"2023-06-28T20:08:21.659501+0000","last_undegraded":"2023-06-28T20:08:21.659501+0000","last_fullsized":"2023-06-28T20:08:21.659501+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:39:02.649739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.658477+0000","last_change":"2023-06-28T20:08:20.664815+0000","last_active":"2023-06-28T20:08:21.658477+0000","last_peered":"2023-06-28T20:08:21.658477+0000","last_clean":"2023-06-28T20:08:21.658477+0000","last_became_active":"2023-06-28T20:08:20.664482+0000","last_became_peered":"2023-06-28T20:08:20.664482+0000","last_unstale":"2023-06-28T20:08:21.658477+0000","last_undegraded":"2023-06-28T20:08:21.658477+0000","last_fullsized":"2023-06-28T20:08:21.658477+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:19:46.143213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108680+0000","last_change":"2023-06-28T20:08:19.663072+0000","last_active":"2023-06-28T20:08:22.108680+0000","last_peered":"2023-06-28T20:08:22.108680+0000","last_clean":"2023-06-28T20:08:22.108680+0000","last_became_active":"2023-06-28T20:08:19.662730+0000","last_became_peered":"2023-06-28T20:08:19.662730+0000","last_unstale":"2023-06-28T20:08:22.108680+0000","last_undegraded":"2023-06-28T20:08:22.108680+0000","last_fullsized":"2023-06-28T20:08:22.108680+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:02:45.086997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":43,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108762+0000","last_change":"2023-06-28T20:07:15.303675+0000","last_active":"2023-06-28T20:08:22.108762+0000","last_peered":"2023-06-28T20:08:22.108762+0000","last_clean":"2023-06-28T20:08:22.108762+0000","last_became_active":"2023-06-28T20:07:15.303350+0000","last_became_peered":"2023-06-28T20:07:15.303350+0000","last_unstale":"2023-06-28T20:08:22.108762+0000","last_undegraded":"2023-06-28T20:08:22.108762+0000","last_fullsized":"2023-06-28T20:08:22.108762+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:12:06.137754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109443+0000","last_change":"2023-06-28T20:08:20.682863+0000","last_active":"2023-06-28T20:08:22.109443+0000","last_peered":"2023-06-28T20:08:22.109443+0000","last_clean":"2023-06-28T20:08:22.109443+0000","last_became_active":"2023-06-28T20:08:20.682499+0000","last_became_peered":"2023-06-28T20:08:20.682499+0000","last_unstale":"2023-06-28T20:08:22.109443+0000","last_undegraded":"2023-06-28T20:08:22.109443+0000","last_fullsized":"2023-06-28T20:08:22.109443+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:56:18.035434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801805+0000","last_change":"2023-06-28T20:08:19.666978+0000","last_active":"2023-06-28T20:08:21.801805+0000","last_peered":"2023-06-28T20:08:21.801805+0000","last_clean":"2023-06-28T20:08:21.801805+0000","last_became_active":"2023-06-28T20:08:19.666613+0000","last_became_peered":"2023-06-28T20:08:19.666613+0000","last_unstale":"2023-06-28T20:08:21.801805+0000","last_undegraded":"2023-06-28T20:08:21.801805+0000","last_fullsized":"2023-06-28T20:08:21.801805+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:22:55.905699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":37,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109348+0000","last_change":"2023-06-28T20:07:15.303565+0000","last_active":"2023-06-28T20:08:22.109348+0000","last_peered":"2023-06-28T20:08:22.109348+0000","last_clean":"2023-06-28T20:08:22.109348+0000","last_became_active":"2023-06-28T20:07:15.303200+0000","last_became_peered":"2023-06-28T20:07:15.303200+0000","last_unstale":"2023-06-28T20:08:22.109348+0000","last_undegraded":"2023-06-28T20:08:22.109348+0000","last_fullsized":"2023-06-28T20:08:22.109348+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:58:08.218147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":185,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659552+0000","last_change":"2023-06-28T20:07:15.304921+0000","last_active":"2023-06-28T20:08:21.659552+0000","last_peered":"2023-06-28T20:08:21.659552+0000","last_clean":"2023-06-28T20:08:21.659552+0000","last_became_active":"2023-06-28T20:07:15.303959+0000","last_became_peered":"2023-06-28T20:07:15.303959+0000","last_unstale":"2023-06-28T20:08:21.659552+0000","last_undegraded":"2023-06-28T20:08:21.659552+0000","last_fullsized":"2023-06-28T20:08:21.659552+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:21:11.722702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660392+0000","last_change":"2023-06-28T20:08:20.667481+0000","last_active":"2023-06-28T20:08:21.660392+0000","last_peered":"2023-06-28T20:08:21.660392+0000","last_clean":"2023-06-28T20:08:21.660392+0000","last_became_active":"2023-06-28T20:08:20.666850+0000","last_became_peered":"2023-06-28T20:08:20.666850+0000","last_unstale":"2023-06-28T20:08:21.660392+0000","last_undegraded":"2023-06-28T20:08:21.660392+0000","last_fullsized":"2023-06-28T20:08:21.660392+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:37:01.278070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"35'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.800649+0000","last_change":"2023-06-28T20:08:19.664561+0000","last_active":"2023-06-28T20:08:21.800649+0000","last_peered":"2023-06-28T20:08:21.800649+0000","last_clean":"2023-06-28T20:08:21.800649+0000","last_became_active":"2023-06-28T20:08:19.663684+0000","last_became_peered":"2023-06-28T20:08:19.663684+0000","last_unstale":"2023-06-28T20:08:21.800649+0000","last_undegraded":"2023-06-28T20:08:21.800649+0000","last_fullsized":"2023-06-28T20:08:21.800649+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:33:29.127873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":37,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.658346+0000","last_change":"2023-06-28T20:07:15.899048+0000","last_active":"2023-06-28T20:08:21.658346+0000","last_peered":"2023-06-28T20:08:21.658346+0000","last_clean":"2023-06-28T20:08:21.658346+0000","last_became_active":"2023-06-28T20:07:15.898766+0000","last_became_peered":"2023-06-28T20:07:15.898766+0000","last_unstale":"2023-06-28T20:08:21.658346+0000","last_undegraded":"2023-06-28T20:08:21.658346+0000","last_fullsized":"2023-06-28T20:08:21.658346+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:04:24.646683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801713+0000","last_change":"2023-06-28T20:08:20.680186+0000","last_active":"2023-06-28T20:08:21.801713+0000","last_peered":"2023-06-28T20:08:21.801713+0000","last_clean":"2023-06-28T20:08:21.801713+0000","last_became_active":"2023-06-28T20:08:20.679936+0000","last_became_peered":"2023-06-28T20:08:20.679936+0000","last_unstale":"2023-06-28T20:08:21.801713+0000","last_undegraded":"2023-06-28T20:08:21.801713+0000","last_fullsized":"2023-06-28T20:08:21.801713+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:20:23.086774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.658977+0000","last_change":"2023-06-28T20:08:19.662452+0000","last_active":"2023-06-28T20:08:21.658977+0000","last_peered":"2023-06-28T20:08:21.658977+0000","last_clean":"2023-06-28T20:08:21.658977+0000","last_became_active":"2023-06-28T20:08:19.661640+0000","last_became_peered":"2023-06-28T20:08:19.661640+0000","last_unstale":"2023-06-28T20:08:21.658977+0000","last_undegraded":"2023-06-28T20:08:21.658977+0000","last_fullsized":"2023-06-28T20:08:21.658977+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:20:22.082165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":35,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659431+0000","last_change":"2023-06-28T20:07:15.899229+0000","last_active":"2023-06-28T20:08:21.659431+0000","last_peered":"2023-06-28T20:08:21.659431+0000","last_clean":"2023-06-28T20:08:21.659431+0000","last_became_active":"2023-06-28T20:07:15.898708+0000","last_became_peered":"2023-06-28T20:07:15.898708+0000","last_unstale":"2023-06-28T20:08:21.659431+0000","last_undegraded":"2023-06-28T20:08:21.659431+0000","last_fullsized":"2023-06-28T20:08:21.659431+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:48:33.012020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108808+0000","last_change":"2023-06-28T20:08:20.674909+0000","last_active":"2023-06-28T20:08:22.108808+0000","last_peered":"2023-06-28T20:08:22.108808+0000","last_clean":"2023-06-28T20:08:22.108808+0000","last_became_active":"2023-06-28T20:08:20.671930+0000","last_became_peered":"2023-06-28T20:08:20.671930+0000","last_unstale":"2023-06-28T20:08:22.108808+0000","last_undegraded":"2023-06-28T20:08:22.108808+0000","last_fullsized":"2023-06-28T20:08:22.108808+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:58:40.898791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109245+0000","last_change":"2023-06-28T20:08:19.663982+0000","last_active":"2023-06-28T20:08:22.109245+0000","last_peered":"2023-06-28T20:08:22.109245+0000","last_clean":"2023-06-28T20:08:22.109245+0000","last_became_active":"2023-06-28T20:08:19.663657+0000","last_became_peered":"2023-06-28T20:08:19.663657+0000","last_unstale":"2023-06-28T20:08:22.109245+0000","last_undegraded":"2023-06-28T20:08:22.109245+0000","last_fullsized":"2023-06-28T20:08:22.109245+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:39:24.497671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801240+0000","last_change":"2023-06-28T20:07:15.898966+0000","last_active":"2023-06-28T20:08:21.801240+0000","last_peered":"2023-06-28T20:08:21.801240+0000","last_clean":"2023-06-28T20:08:21.801240+0000","last_became_active":"2023-06-28T20:07:15.898704+0000","last_became_peered":"2023-06-28T20:07:15.898704+0000","last_unstale":"2023-06-28T20:08:21.801240+0000","last_undegraded":"2023-06-28T20:08:21.801240+0000","last_fullsized":"2023-06-28T20:08:21.801240+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:58:41.892167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108131+0000","last_change":"2023-06-28T20:08:20.667824+0000","last_active":"2023-06-28T20:08:22.108131+0000","last_peered":"2023-06-28T20:08:22.108131+0000","last_clean":"2023-06-28T20:08:22.108131+0000","last_became_active":"2023-06-28T20:08:20.667548+0000","last_became_peered":"2023-06-28T20:08:20.667548+0000","last_unstale":"2023-06-28T20:08:22.108131+0000","last_undegraded":"2023-06-28T20:08:22.108131+0000","last_fullsized":"2023-06-28T20:08:22.108131+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:11:02.786663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660615+0000","last_change":"2023-06-28T20:08:19.653324+0000","last_active":"2023-06-28T20:08:21.660615+0000","last_peered":"2023-06-28T20:08:21.660615+0000","last_clean":"2023-06-28T20:08:21.660615+0000","last_became_active":"2023-06-28T20:08:19.652941+0000","last_became_peered":"2023-06-28T20:08:19.652941+0000","last_unstale":"2023-06-28T20:08:21.660615+0000","last_undegraded":"2023-06-28T20:08:21.660615+0000","last_fullsized":"2023-06-28T20:08:21.660615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:20:22.082165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":47,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660032+0000","last_change":"2023-06-28T20:07:15.899207+0000","last_active":"2023-06-28T20:08:21.660032+0000","last_peered":"2023-06-28T20:08:21.660032+0000","last_clean":"2023-06-28T20:08:21.660032+0000","last_became_active":"2023-06-28T20:07:15.898908+0000","last_became_peered":"2023-06-28T20:07:15.898908+0000","last_unstale":"2023-06-28T20:08:21.660032+0000","last_undegraded":"2023-06-28T20:08:21.660032+0000","last_fullsized":"2023-06-28T20:08:21.660032+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:57:10.381253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109908+0000","last_change":"2023-06-28T20:08:20.687344+0000","last_active":"2023-06-28T20:08:22.109908+0000","last_peered":"2023-06-28T20:08:22.109908+0000","last_clean":"2023-06-28T20:08:22.109908+0000","last_became_active":"2023-06-28T20:08:20.687061+0000","last_became_peered":"2023-06-28T20:08:20.687061+0000","last_unstale":"2023-06-28T20:08:22.109908+0000","last_undegraded":"2023-06-28T20:08:22.109908+0000","last_fullsized":"2023-06-28T20:08:22.109908+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:15:25.870011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108612+0000","last_change":"2023-06-28T20:08:19.650567+0000","last_active":"2023-06-28T20:08:22.108612+0000","last_peered":"2023-06-28T20:08:22.108612+0000","last_clean":"2023-06-28T20:08:22.108612+0000","last_became_active":"2023-06-28T20:08:19.650333+0000","last_became_peered":"2023-06-28T20:08:19.650333+0000","last_unstale":"2023-06-28T20:08:22.108612+0000","last_undegraded":"2023-06-28T20:08:22.108612+0000","last_fullsized":"2023-06-28T20:08:22.108612+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:56:17.030824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":47,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660200+0000","last_change":"2023-06-28T20:07:15.303873+0000","last_active":"2023-06-28T20:08:21.660200+0000","last_peered":"2023-06-28T20:08:21.660200+0000","last_clean":"2023-06-28T20:08:21.660200+0000","last_became_active":"2023-06-28T20:07:15.303457+0000","last_became_peered":"2023-06-28T20:07:15.303457+0000","last_unstale":"2023-06-28T20:08:21.660200+0000","last_undegraded":"2023-06-28T20:08:21.660200+0000","last_fullsized":"2023-06-28T20:08:21.660200+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:27:15.184713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":1007616,"data_stored":756976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.75},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.71099999999999997}]},{"osd":1,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.59499999999999997}]},{"osd":2,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.67100000000000004}]},{"osd":3,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.51600000000000001}]},{"osd":4,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.46200000000000002}]},{"osd":5,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.434},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.54900000000000004}]},{"osd":6,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.52900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":401408,"data_stored":165556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":9.0069999999999997,"5min":9.0069999999999997,"15min":9.0069999999999997},"last":0.58799999999999997}]},{"osd":1,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":3.036,"5min":3.036,"15min":3.036},"last":0.625},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.60899999999999999}]},{"osd":2,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.66900000000000004}]},{"osd":3,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":9.0459999999999994,"5min":9.0459999999999994,"15min":9.0459999999999994},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.69799999999999995}]},{"osd":4,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.42299999999999999}]},{"osd":5,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.51200000000000001}]},{"osd":7,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.53200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":995328,"data_stored":755432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.26500000000000001}]},{"osd":2,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.34699999999999998}]},{"osd":3,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.36799999999999999}]},{"osd":4,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56599999999999995}]},{"osd":5,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59599999999999997}]},{"osd":6,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.497},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.624}]},{"osd":7,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":397312,"data_stored":164047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.219}]},{"osd":2,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.442,"5min":0.442,"15min":0.442},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.26600000000000001}]},{"osd":3,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.28499999999999998}]},{"osd":4,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.434},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.38700000000000001}]},{"osd":5,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.39400000000000002}]},{"osd":6,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.54500000000000004}]},{"osd":7,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.47199999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":401408,"data_stored":165044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":16.044,"5min":16.044,"15min":16.044},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.34499999999999997}]},{"osd":1,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":11.028,"5min":11.028,"15min":11.028},"last":0.27700000000000002},{"interface":"front","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":16.077000000000002,"5min":16.077000000000002,"15min":16.077000000000002},"last":0.42599999999999999}]},{"osd":3,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.36499999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.319,"15min":1.319},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":16.004000000000001,"5min":16.004000000000001,"15min":16.004000000000001},"last":0.38100000000000001}]},{"osd":4,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":9.327,"5min":9.327,"15min":9.327},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":10.964,"5min":10.964,"15min":10.964},"last":0.86499999999999999}]},{"osd":5,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":10.923,"5min":10.923,"15min":10.923},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.81699999999999995}]},{"osd":6,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":9.282,"5min":9.282,"15min":9.282},"last":0.59099999999999997}]},{"osd":7,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":10.996,"5min":10.996,"15min":10.996},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.76700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":166572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":5.0250000000000004,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":0.46600000000000003}]},{"osd":1,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":5.2309999999999999,"5min":5.2309999999999999,"15min":5.2309999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":5.3250000000000002,"5min":5.3250000000000002,"15min":5.3250000000000002},"last":0.40100000000000002}]},{"osd":2,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":15.927,"5min":15.927,"15min":15.927},"last":0.44},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.36299999999999999}]},{"osd":4,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.5},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.52500000000000002}]},{"osd":5,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":4.9939999999999998,"5min":4.9939999999999998,"15min":4.9939999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.65400000000000003}]},{"osd":6,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":4.9619999999999997,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.59499999999999997}]},{"osd":7,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":4.9210000000000003,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.626}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":405504,"data_stored":165578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":9.1809999999999992,"5min":9.1809999999999992,"15min":9.1809999999999992},"last":0.39900000000000002}]},{"osd":1,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.627}]},{"osd":2,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":9.0700000000000003,"5min":9.0700000000000003,"15min":9.0700000000000003},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.64400000000000002}]},{"osd":3,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.60499999999999998}]},{"osd":5,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.753}]},{"osd":6,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.73599999999999999}]},{"osd":7,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.77200000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":405504,"data_stored":165086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.45200000000000001}]},{"osd":1,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.47799999999999998}]},{"osd":2,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.60799999999999998}]},{"osd":3,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":5.6139999999999999,"5min":5.6139999999999999,"15min":5.6139999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":5.6500000000000004,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.52600000000000002}]},{"osd":4,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.54800000000000004}]},{"osd":6,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.5},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.69799999999999995}]},{"osd":7,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.63600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-28T20:08:25.138 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-28T20:08:25.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.291+0000 7f9f0fc78700 1 Processor -- start 2023-06-28T20:08:25.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.291+0000 7f9f0fc78700 1 -- start start 2023-06-28T20:08:25.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.292+0000 7f9f0fc78700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f080af060 0x7f9f080cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:25.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.292+0000 7f9f0fc78700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9f0806ca80 0x7f9f08069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:25.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.292+0000 7f9f0fc78700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f08071410 con 0x7f9f0806ce80 2023-06-28T20:08:25.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.292+0000 7f9f0fc78700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f080716b0 con 0x7f9f0806ca80 2023-06-28T20:08:25.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.292+0000 7f9f0fc78700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f08071990 con 0x7f9f080af060 2023-06-28T20:08:25.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.292+0000 7f9f0d213700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9f0806ca80 0x7f9f08069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:25.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.292+0000 7f9f0da14700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f080af060 0x7f9f080cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:25.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.292+0000 7f9f0d213700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f9f0806ca80 0x7f9f08069ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:41118/0 (socket says 172.21.15.43:41118) 2023-06-28T20:08:25.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.292+0000 7f9f0da14700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f080af060 0x7f9f080cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.43:39026/0 (socket says 172.21.15.43:39026) 2023-06-28T20:08:25.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.292+0000 7f9f0d213700 1 -- 172.21.15.43:0/4012119192 learned_addr learned my addr 172.21.15.43:0/4012119192 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:25.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.292+0000 7f9f0e215700 1 --1- 172.21.15.43:0/4012119192 >> v1:172.21.15.178:6789/0 conn(0x7f9f0806ce80 0x7f9f080ac870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:43368/0 (socket says 172.21.15.43:43368) 2023-06-28T20:08:25.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.293+0000 7f9f0d213700 1 -- 172.21.15.43:0/4012119192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f080af060 msgr2=0x7f9f080cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:25.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.293+0000 7f9f0d213700 1 --2- 172.21.15.43:0/4012119192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f080af060 0x7f9f080cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.293+0000 7f9f0d213700 1 -- 172.21.15.43:0/4012119192 >> v1:172.21.15.178:6789/0 conn(0x7f9f0806ce80 legacy=0x7f9f080ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:25.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.293+0000 7f9f0d213700 1 -- 172.21.15.43:0/4012119192 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f080af430 con 0x7f9f0806ca80 2023-06-28T20:08:25.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.293+0000 7f9f0ca12700 1 -- 172.21.15.43:0/4012119192 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429929656 0 0) 0x7f9f08071410 con 0x7f9f0806ce80 2023-06-28T20:08:25.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.293+0000 7f9f0da14700 1 --2- 172.21.15.43:0/4012119192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f080af060 0x7f9f080cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:08:25.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.293+0000 7f9f0d213700 1 --2- 172.21.15.43:0/4012119192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f0806ca80 0x7f9f08069ef0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9ef80098f0 tx=0x7f9ef8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67c821b169194482 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:25.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.293+0000 7f9f0ca12700 1 -- 172.21.15.43:0/4012119192 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8011830 con 0x7f9f0806ca80 2023-06-28T20:08:25.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.293+0000 7f9f0ca12700 1 -- 172.21.15.43:0/4012119192 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef8011990 con 0x7f9f0806ca80 2023-06-28T20:08:25.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.293+0000 7f9f0ca12700 1 -- 172.21.15.43:0/4012119192 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8011d20 con 0x7f9f0806ca80 2023-06-28T20:08:25.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.294+0000 7f9f0fc78700 1 -- 172.21.15.43:0/4012119192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f0806ca80 msgr2=0x7f9f08069ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:25.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.294+0000 7f9f0fc78700 1 --2- 172.21.15.43:0/4012119192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f0806ca80 0x7f9f08069ef0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9ef80098f0 tx=0x7f9ef8009c00 comp rx=0 tx=0).stop 2023-06-28T20:08:25.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.294+0000 7f9f0fc78700 1 -- 172.21.15.43:0/4012119192 shutdown_connections 2023-06-28T20:08:25.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.294+0000 7f9f0fc78700 1 --2- 172.21.15.43:0/4012119192 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f080af060 0x7f9f080cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.294+0000 7f9f0fc78700 1 --2- 172.21.15.43:0/4012119192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f0806ca80 0x7f9f08069ef0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.294+0000 7f9f0fc78700 1 -- 172.21.15.43:0/4012119192 >> 172.21.15.43:0/4012119192 conn(0x7f9f080b1720 msgr2=0x7f9f080b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:25.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.294+0000 7f9f0fc78700 1 -- 172.21.15.43:0/4012119192 shutdown_connections 2023-06-28T20:08:25.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.294+0000 7f9f0fc78700 1 -- 172.21.15.43:0/4012119192 wait complete. 2023-06-28T20:08:25.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.295+0000 7f9f0fc78700 1 Processor -- start 2023-06-28T20:08:25.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.295+0000 7f9f0fc78700 1 -- start start 2023-06-28T20:08:25.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.296+0000 7f9f0fc78700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f0806ce80 0x7f9f0815e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:25.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.296+0000 7f9f0fc78700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f080af060 0x7f9f08158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:25.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.296+0000 7f9f0fc78700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f08157ce0 0x7f9f080c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:25.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.296+0000 7f9f0fc78700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f08159880 con 0x7f9f080af060 2023-06-28T20:08:25.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.296+0000 7f9f0fc78700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0815da40 con 0x7f9f0806ce80 2023-06-28T20:08:25.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.296+0000 7f9f0fc78700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f08158c00 con 0x7f9f08157ce0 2023-06-28T20:08:25.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.296+0000 7f9f0d213700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f080af060 0x7f9f08158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:25.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.296+0000 7f9f0d213700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f080af060 0x7f9f08158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55062/0 (socket says 172.21.15.43:55062) 2023-06-28T20:08:25.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.296+0000 7f9f0d213700 1 -- 172.21.15.43:0/3626611366 learned_addr learned my addr 172.21.15.43:0/3626611366 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:25.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.296+0000 7f9f0da14700 1 --2- 172.21.15.43:0/3626611366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f0806ce80 0x7f9f0815e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:25.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.296+0000 7f9f0e215700 1 --2- 172.21.15.43:0/3626611366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f08157ce0 0x7f9f080c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:25.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.296+0000 7f9f0d213700 1 -- 172.21.15.43:0/3626611366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f08157ce0 msgr2=0x7f9f080c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:25.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.296+0000 7f9f0d213700 1 --2- 172.21.15.43:0/3626611366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f08157ce0 0x7f9f080c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.296+0000 7f9f0d213700 1 -- 172.21.15.43:0/3626611366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f0806ce80 msgr2=0x7f9f0815e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:25.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.296+0000 7f9f0d213700 1 --2- 172.21.15.43:0/3626611366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f0806ce80 0x7f9f0815e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.296+0000 7f9f0d213700 1 -- 172.21.15.43:0/3626611366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f040094a0 con 0x7f9f080af060 2023-06-28T20:08:25.315 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.296+0000 7f9f0da14700 1 --2- 172.21.15.43:0/3626611366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f0806ce80 0x7f9f0815e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:25.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.296+0000 7f9f0d213700 1 --2- 172.21.15.43:0/3626611366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f080af060 0x7f9f08158500 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9f08072270 tx=0x7f9ef8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:25.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.296+0000 7f9efe7fc700 1 -- 172.21.15.43:0/3626611366 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef80044b0 con 0x7f9f080af060 2023-06-28T20:08:25.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.296+0000 7f9efe7fc700 1 -- 172.21.15.43:0/3626611366 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef801f070 con 0x7f9f080af060 2023-06-28T20:08:25.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.296+0000 7f9f0fc78700 1 -- 172.21.15.43:0/3626611366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef8009550 con 0x7f9f080af060 2023-06-28T20:08:25.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.297+0000 7f9efe7fc700 1 -- 172.21.15.43:0/3626611366 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef80038a0 con 0x7f9f080af060 2023-06-28T20:08:25.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.297+0000 7f9f0fc78700 1 -- 172.21.15.43:0/3626611366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f081591c0 con 0x7f9f080af060 2023-06-28T20:08:25.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.297+0000 7f9f0fc78700 1 -- 172.21.15.43:0/3626611366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f080728b0 con 0x7f9f080af060 2023-06-28T20:08:25.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.298+0000 7f9efe7fc700 1 -- 172.21.15.43:0/3626611366 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9ef8021030 con 0x7f9f080af060 2023-06-28T20:08:25.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.298+0000 7f9efe7fc700 1 --2- 172.21.15.43:0/3626611366 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9eec06ace0 0x7f9eec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:25.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.298+0000 7f9efe7fc700 1 -- 172.21.15.43:0/3626611366 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9ef808aab0 con 0x7f9f080af060 2023-06-28T20:08:25.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.298+0000 7f9f0da14700 1 --2- 172.21.15.43:0/3626611366 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9eec06ace0 0x7f9eec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:25.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.299+0000 7f9f0da14700 1 --2- 172.21.15.43:0/3626611366 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9eec06ace0 0x7f9eec06d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009d20 tx=0x7f9f04008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:25.321 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.303+0000 7f9efe7fc700 1 -- 172.21.15.43:0/3626611366 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9ef8055c90 con 0x7f9f080af060 2023-06-28T20:08:25.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.495+0000 7f9f0fc78700 1 -- 172.21.15.43:0/3626611366 --> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9f0806db00 con 0x7f9eec06ace0 2023-06-28T20:08:25.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.504+0000 7f9efe7fc700 1 -- 172.21.15.43:0/3626611366 <== mgr.4101 v2:172.21.15.43:6832/406144153 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311484 (secure 0 0 0) 0x7f9f04009970 con 0x7f9eec06ace0 2023-06-28T20:08:25.511 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:25.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.508+0000 7f9f0fc78700 1 -- 172.21.15.43:0/3626611366 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9eec06ace0 msgr2=0x7f9eec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:25.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.508+0000 7f9f0fc78700 1 --2- 172.21.15.43:0/3626611366 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9eec06ace0 0x7f9eec06d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009d20 tx=0x7f9f04008040 comp rx=0 tx=0).stop 2023-06-28T20:08:25.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.508+0000 7f9f0fc78700 1 -- 172.21.15.43:0/3626611366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f080af060 msgr2=0x7f9f08158500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:25.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.508+0000 7f9f0fc78700 1 --2- 172.21.15.43:0/3626611366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f080af060 0x7f9f08158500 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9f08072270 tx=0x7f9ef8004750 comp rx=0 tx=0).stop 2023-06-28T20:08:25.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.508+0000 7f9f0fc78700 1 -- 172.21.15.43:0/3626611366 shutdown_connections 2023-06-28T20:08:25.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.508+0000 7f9f0fc78700 1 --2- 172.21.15.43:0/3626611366 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f9eec06ace0 0x7f9eec06d190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.508+0000 7f9f0fc78700 1 --2- 172.21.15.43:0/3626611366 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f9f08157ce0 0x7f9f080c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.508+0000 7f9f0fc78700 1 --2- 172.21.15.43:0/3626611366 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f0806ce80 0x7f9f0815e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.508+0000 7f9f0fc78700 1 --2- 172.21.15.43:0/3626611366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f080af060 0x7f9f08158500 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.508+0000 7f9f0fc78700 1 -- 172.21.15.43:0/3626611366 >> 172.21.15.43:0/3626611366 conn(0x7f9f080b1720 msgr2=0x7f9f080afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:25.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.508+0000 7f9f0fc78700 1 -- 172.21.15.43:0/3626611366 shutdown_connections 2023-06-28T20:08:25.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.508+0000 7f9f0fc78700 1 -- 172.21.15.43:0/3626611366 wait complete. 2023-06-28T20:08:25.515 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-06-28T20:08:25.524 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2023-06-28T20:08:24.003794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":163,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219940,"kb_used_data":4324,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754780,"statfs":{"total":773094113280,"available":772868894720,"internally_reserved":0,"allocated":4427776,"data_stored":2504291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.362094"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109380+0000","last_change":"2023-06-28T20:08:20.679268+0000","last_active":"2023-06-28T20:08:22.109380+0000","last_peered":"2023-06-28T20:08:22.109380+0000","last_clean":"2023-06-28T20:08:22.109380+0000","last_became_active":"2023-06-28T20:08:20.678933+0000","last_became_peered":"2023-06-28T20:08:20.678933+0000","last_unstale":"2023-06-28T20:08:22.109380+0000","last_undegraded":"2023-06-28T20:08:22.109380+0000","last_fullsized":"2023-06-28T20:08:22.109380+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:01:15.843032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108652+0000","last_change":"2023-06-28T20:08:20.685759+0000","last_active":"2023-06-28T20:08:22.108652+0000","last_peered":"2023-06-28T20:08:22.108652+0000","last_clean":"2023-06-28T20:08:22.108652+0000","last_became_active":"2023-06-28T20:08:20.685463+0000","last_became_peered":"2023-06-28T20:08:20.685463+0000","last_unstale":"2023-06-28T20:08:22.108652+0000","last_undegraded":"2023-06-28T20:08:22.108652+0000","last_fullsized":"2023-06-28T20:08:22.108652+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:09:08.348528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108848+0000","last_change":"2023-06-28T20:08:20.680151+0000","last_active":"2023-06-28T20:08:22.108848+0000","last_peered":"2023-06-28T20:08:22.108848+0000","last_clean":"2023-06-28T20:08:22.108848+0000","last_became_active":"2023-06-28T20:08:20.679855+0000","last_became_peered":"2023-06-28T20:08:20.679855+0000","last_unstale":"2023-06-28T20:08:22.108848+0000","last_undegraded":"2023-06-28T20:08:22.108848+0000","last_fullsized":"2023-06-28T20:08:22.108848+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:17:11.674363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.799185+0000","last_change":"2023-06-28T20:08:20.685562+0000","last_active":"2023-06-28T20:08:21.799185+0000","last_peered":"2023-06-28T20:08:21.799185+0000","last_clean":"2023-06-28T20:08:21.799185+0000","last_became_active":"2023-06-28T20:08:20.684942+0000","last_became_peered":"2023-06-28T20:08:20.684942+0000","last_unstale":"2023-06-28T20:08:21.799185+0000","last_undegraded":"2023-06-28T20:08:21.799185+0000","last_fullsized":"2023-06-28T20:08:21.799185+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:06:33.836982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.798759+0000","last_change":"2023-06-28T20:08:20.679584+0000","last_active":"2023-06-28T20:08:21.798759+0000","last_peered":"2023-06-28T20:08:21.798759+0000","last_clean":"2023-06-28T20:08:21.798759+0000","last_became_active":"2023-06-28T20:08:20.679308+0000","last_became_peered":"2023-06-28T20:08:20.679308+0000","last_unstale":"2023-06-28T20:08:21.798759+0000","last_undegraded":"2023-06-28T20:08:21.798759+0000","last_fullsized":"2023-06-28T20:08:21.798759+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:32:28.393764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109307+0000","last_change":"2023-06-28T20:08:20.678092+0000","last_active":"2023-06-28T20:08:22.109307+0000","last_peered":"2023-06-28T20:08:22.109307+0000","last_clean":"2023-06-28T20:08:22.109307+0000","last_became_active":"2023-06-28T20:08:20.677783+0000","last_became_peered":"2023-06-28T20:08:20.677783+0000","last_unstale":"2023-06-28T20:08:22.109307+0000","last_undegraded":"2023-06-28T20:08:22.109307+0000","last_fullsized":"2023-06-28T20:08:22.109307+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:52:01.183214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801212+0000","last_change":"2023-06-28T20:08:20.683460+0000","last_active":"2023-06-28T20:08:21.801212+0000","last_peered":"2023-06-28T20:08:21.801212+0000","last_clean":"2023-06-28T20:08:21.801212+0000","last_became_active":"2023-06-28T20:08:20.682294+0000","last_became_peered":"2023-06-28T20:08:20.682294+0000","last_unstale":"2023-06-28T20:08:21.801212+0000","last_undegraded":"2023-06-28T20:08:21.801212+0000","last_fullsized":"2023-06-28T20:08:21.801212+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:47:32.289655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108529+0000","last_change":"2023-06-28T20:08:20.680634+0000","last_active":"2023-06-28T20:08:22.108529+0000","last_peered":"2023-06-28T20:08:22.108529+0000","last_clean":"2023-06-28T20:08:22.108529+0000","last_became_active":"2023-06-28T20:08:20.679711+0000","last_became_peered":"2023-06-28T20:08:20.679711+0000","last_unstale":"2023-06-28T20:08:22.108529+0000","last_undegraded":"2023-06-28T20:08:22.108529+0000","last_fullsized":"2023-06-28T20:08:22.108529+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:09:34.429733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108391+0000","last_change":"2023-06-28T20:08:20.687504+0000","last_active":"2023-06-28T20:08:22.108391+0000","last_peered":"2023-06-28T20:08:22.108391+0000","last_clean":"2023-06-28T20:08:22.108391+0000","last_became_active":"2023-06-28T20:08:20.686571+0000","last_became_peered":"2023-06-28T20:08:20.686571+0000","last_unstale":"2023-06-28T20:08:22.108391+0000","last_undegraded":"2023-06-28T20:08:22.108391+0000","last_fullsized":"2023-06-28T20:08:22.108391+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:59:13.572447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801525+0000","last_change":"2023-06-28T20:08:20.682066+0000","last_active":"2023-06-28T20:08:21.801525+0000","last_peered":"2023-06-28T20:08:21.801525+0000","last_clean":"2023-06-28T20:08:21.801525+0000","last_became_active":"2023-06-28T20:08:20.681764+0000","last_became_peered":"2023-06-28T20:08:20.681764+0000","last_unstale":"2023-06-28T20:08:21.801525+0000","last_undegraded":"2023-06-28T20:08:21.801525+0000","last_fullsized":"2023-06-28T20:08:21.801525+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:59:28.529049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801518+0000","last_change":"2023-06-28T20:08:20.678239+0000","last_active":"2023-06-28T20:08:21.801518+0000","last_peered":"2023-06-28T20:08:21.801518+0000","last_clean":"2023-06-28T20:08:21.801518+0000","last_became_active":"2023-06-28T20:08:20.677233+0000","last_became_peered":"2023-06-28T20:08:20.677233+0000","last_unstale":"2023-06-28T20:08:21.801518+0000","last_undegraded":"2023-06-28T20:08:21.801518+0000","last_fullsized":"2023-06-28T20:08:21.801518+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:28:20.539013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108313+0000","last_change":"2023-06-28T20:08:20.686755+0000","last_active":"2023-06-28T20:08:22.108313+0000","last_peered":"2023-06-28T20:08:22.108313+0000","last_clean":"2023-06-28T20:08:22.108313+0000","last_became_active":"2023-06-28T20:08:20.686386+0000","last_became_peered":"2023-06-28T20:08:20.686386+0000","last_unstale":"2023-06-28T20:08:22.108313+0000","last_undegraded":"2023-06-28T20:08:22.108313+0000","last_fullsized":"2023-06-28T20:08:22.108313+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:05:46.974291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801475+0000","last_change":"2023-06-28T20:08:20.683528+0000","last_active":"2023-06-28T20:08:21.801475+0000","last_peered":"2023-06-28T20:08:21.801475+0000","last_clean":"2023-06-28T20:08:21.801475+0000","last_became_active":"2023-06-28T20:08:20.681838+0000","last_became_peered":"2023-06-28T20:08:20.681838+0000","last_unstale":"2023-06-28T20:08:21.801475+0000","last_undegraded":"2023-06-28T20:08:21.801475+0000","last_fullsized":"2023-06-28T20:08:21.801475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:21:39.943734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659709+0000","last_change":"2023-06-28T20:08:20.666405+0000","last_active":"2023-06-28T20:08:21.659709+0000","last_peered":"2023-06-28T20:08:21.659709+0000","last_clean":"2023-06-28T20:08:21.659709+0000","last_became_active":"2023-06-28T20:08:20.666130+0000","last_became_peered":"2023-06-28T20:08:20.666130+0000","last_unstale":"2023-06-28T20:08:21.659709+0000","last_undegraded":"2023-06-28T20:08:21.659709+0000","last_fullsized":"2023-06-28T20:08:21.659709+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:32:32.975208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801071+0000","last_change":"2023-06-28T20:08:20.687883+0000","last_active":"2023-06-28T20:08:21.801071+0000","last_peered":"2023-06-28T20:08:21.801071+0000","last_clean":"2023-06-28T20:08:21.801071+0000","last_became_active":"2023-06-28T20:08:20.687583+0000","last_became_peered":"2023-06-28T20:08:20.687583+0000","last_unstale":"2023-06-28T20:08:21.801071+0000","last_undegraded":"2023-06-28T20:08:21.801071+0000","last_fullsized":"2023-06-28T20:08:21.801071+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:52:36.907231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.658620+0000","last_change":"2023-06-28T20:08:20.678870+0000","last_active":"2023-06-28T20:08:21.658620+0000","last_peered":"2023-06-28T20:08:21.658620+0000","last_clean":"2023-06-28T20:08:21.658620+0000","last_became_active":"2023-06-28T20:08:20.678608+0000","last_became_peered":"2023-06-28T20:08:20.678608+0000","last_unstale":"2023-06-28T20:08:21.658620+0000","last_undegraded":"2023-06-28T20:08:21.658620+0000","last_fullsized":"2023-06-28T20:08:21.658620+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:28:20.539013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660565+0000","last_change":"2023-06-28T20:08:20.679525+0000","last_active":"2023-06-28T20:08:21.660565+0000","last_peered":"2023-06-28T20:08:21.660565+0000","last_clean":"2023-06-28T20:08:21.660565+0000","last_became_active":"2023-06-28T20:08:20.679222+0000","last_became_peered":"2023-06-28T20:08:20.679222+0000","last_unstale":"2023-06-28T20:08:21.660565+0000","last_undegraded":"2023-06-28T20:08:21.660565+0000","last_fullsized":"2023-06-28T20:08:21.660565+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:12:22.784860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660002+0000","last_change":"2023-06-28T20:08:20.667998+0000","last_active":"2023-06-28T20:08:21.660002+0000","last_peered":"2023-06-28T20:08:21.660002+0000","last_clean":"2023-06-28T20:08:21.660002+0000","last_became_active":"2023-06-28T20:08:20.665959+0000","last_became_peered":"2023-06-28T20:08:20.665959+0000","last_unstale":"2023-06-28T20:08:21.660002+0000","last_undegraded":"2023-06-28T20:08:21.660002+0000","last_fullsized":"2023-06-28T20:08:21.660002+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:15:54.670164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.798909+0000","last_change":"2023-06-28T20:08:20.687229+0000","last_active":"2023-06-28T20:08:21.798909+0000","last_peered":"2023-06-28T20:08:21.798909+0000","last_clean":"2023-06-28T20:08:21.798909+0000","last_became_active":"2023-06-28T20:08:20.686917+0000","last_became_peered":"2023-06-28T20:08:20.686917+0000","last_unstale":"2023-06-28T20:08:21.798909+0000","last_undegraded":"2023-06-28T20:08:21.798909+0000","last_fullsized":"2023-06-28T20:08:21.798909+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:01:55.475356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660191+0000","last_change":"2023-06-28T20:08:20.666513+0000","last_active":"2023-06-28T20:08:21.660191+0000","last_peered":"2023-06-28T20:08:21.660191+0000","last_clean":"2023-06-28T20:08:21.660191+0000","last_became_active":"2023-06-28T20:08:20.666222+0000","last_became_peered":"2023-06-28T20:08:20.666222+0000","last_unstale":"2023-06-28T20:08:21.660191+0000","last_undegraded":"2023-06-28T20:08:21.660191+0000","last_fullsized":"2023-06-28T20:08:21.660191+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:00:06.813246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801519+0000","last_change":"2023-06-28T20:08:20.679222+0000","last_active":"2023-06-28T20:08:21.801519+0000","last_peered":"2023-06-28T20:08:21.801519+0000","last_clean":"2023-06-28T20:08:21.801519+0000","last_became_active":"2023-06-28T20:08:20.678940+0000","last_became_peered":"2023-06-28T20:08:20.678940+0000","last_unstale":"2023-06-28T20:08:21.801519+0000","last_undegraded":"2023-06-28T20:08:21.801519+0000","last_fullsized":"2023-06-28T20:08:21.801519+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:00:06.813246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659165+0000","last_change":"2023-06-28T20:08:20.684209+0000","last_active":"2023-06-28T20:08:21.659165+0000","last_peered":"2023-06-28T20:08:21.659165+0000","last_clean":"2023-06-28T20:08:21.659165+0000","last_became_active":"2023-06-28T20:08:20.683888+0000","last_became_peered":"2023-06-28T20:08:20.683888+0000","last_unstale":"2023-06-28T20:08:21.659165+0000","last_undegraded":"2023-06-28T20:08:21.659165+0000","last_fullsized":"2023-06-28T20:08:21.659165+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:12:02.588448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659187+0000","last_change":"2023-06-28T20:08:20.685467+0000","last_active":"2023-06-28T20:08:21.659187+0000","last_peered":"2023-06-28T20:08:21.659187+0000","last_clean":"2023-06-28T20:08:21.659187+0000","last_became_active":"2023-06-28T20:08:20.685099+0000","last_became_peered":"2023-06-28T20:08:20.685099+0000","last_unstale":"2023-06-28T20:08:21.659187+0000","last_undegraded":"2023-06-28T20:08:21.659187+0000","last_fullsized":"2023-06-28T20:08:21.659187+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:42:21.632954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.798926+0000","last_change":"2023-06-28T20:08:20.685718+0000","last_active":"2023-06-28T20:08:21.798926+0000","last_peered":"2023-06-28T20:08:21.798926+0000","last_clean":"2023-06-28T20:08:21.798926+0000","last_became_active":"2023-06-28T20:08:20.684773+0000","last_became_peered":"2023-06-28T20:08:20.684773+0000","last_unstale":"2023-06-28T20:08:21.798926+0000","last_undegraded":"2023-06-28T20:08:21.798926+0000","last_fullsized":"2023-06-28T20:08:21.798926+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:56:30.427117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660102+0000","last_change":"2023-06-28T20:08:20.676377+0000","last_active":"2023-06-28T20:08:21.660102+0000","last_peered":"2023-06-28T20:08:21.660102+0000","last_clean":"2023-06-28T20:08:21.660102+0000","last_became_active":"2023-06-28T20:08:20.675433+0000","last_became_peered":"2023-06-28T20:08:20.675433+0000","last_unstale":"2023-06-28T20:08:21.660102+0000","last_undegraded":"2023-06-28T20:08:21.660102+0000","last_fullsized":"2023-06-28T20:08:21.660102+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:19:46.143213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109555+0000","last_change":"2023-06-28T20:08:20.679204+0000","last_active":"2023-06-28T20:08:22.109555+0000","last_peered":"2023-06-28T20:08:22.109555+0000","last_clean":"2023-06-28T20:08:22.109555+0000","last_became_active":"2023-06-28T20:08:20.678942+0000","last_became_peered":"2023-06-28T20:08:20.678942+0000","last_unstale":"2023-06-28T20:08:22.109555+0000","last_undegraded":"2023-06-28T20:08:22.109555+0000","last_fullsized":"2023-06-28T20:08:22.109555+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:17:11.674363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660081+0000","last_change":"2023-06-28T20:08:20.687837+0000","last_active":"2023-06-28T20:08:21.660081+0000","last_peered":"2023-06-28T20:08:21.660081+0000","last_clean":"2023-06-28T20:08:21.660081+0000","last_became_active":"2023-06-28T20:08:20.687372+0000","last_became_peered":"2023-06-28T20:08:20.687372+0000","last_unstale":"2023-06-28T20:08:21.660081+0000","last_undegraded":"2023-06-28T20:08:21.660081+0000","last_fullsized":"2023-06-28T20:08:21.660081+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:05:30.000982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801377+0000","last_change":"2023-06-28T20:08:20.689285+0000","last_active":"2023-06-28T20:08:21.801377+0000","last_peered":"2023-06-28T20:08:21.801377+0000","last_clean":"2023-06-28T20:08:21.801377+0000","last_became_active":"2023-06-28T20:08:20.689030+0000","last_became_peered":"2023-06-28T20:08:20.689030+0000","last_unstale":"2023-06-28T20:08:21.801377+0000","last_undegraded":"2023-06-28T20:08:21.801377+0000","last_fullsized":"2023-06-28T20:08:21.801377+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:05:30.000982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659542+0000","last_change":"2023-06-28T20:08:20.667476+0000","last_active":"2023-06-28T20:08:21.659542+0000","last_peered":"2023-06-28T20:08:21.659542+0000","last_clean":"2023-06-28T20:08:21.659542+0000","last_became_active":"2023-06-28T20:08:20.667132+0000","last_became_peered":"2023-06-28T20:08:20.667132+0000","last_unstale":"2023-06-28T20:08:21.659542+0000","last_undegraded":"2023-06-28T20:08:21.659542+0000","last_fullsized":"2023-06-28T20:08:21.659542+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:21:39.943734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660362+0000","last_change":"2023-06-28T20:08:20.667291+0000","last_active":"2023-06-28T20:08:21.660362+0000","last_peered":"2023-06-28T20:08:21.660362+0000","last_clean":"2023-06-28T20:08:21.660362+0000","last_became_active":"2023-06-28T20:08:20.666522+0000","last_became_peered":"2023-06-28T20:08:20.666522+0000","last_unstale":"2023-06-28T20:08:21.660362+0000","last_undegraded":"2023-06-28T20:08:21.660362+0000","last_fullsized":"2023-06-28T20:08:21.660362+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:57:33.386641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.658267+0000","last_change":"2023-06-28T20:08:20.680130+0000","last_active":"2023-06-28T20:08:21.658267+0000","last_peered":"2023-06-28T20:08:21.658267+0000","last_clean":"2023-06-28T20:08:21.658267+0000","last_became_active":"2023-06-28T20:08:20.679865+0000","last_became_peered":"2023-06-28T20:08:20.679865+0000","last_unstale":"2023-06-28T20:08:21.658267+0000","last_undegraded":"2023-06-28T20:08:21.658267+0000","last_fullsized":"2023-06-28T20:08:21.658267+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:59:47.246466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108041+0000","last_change":"2023-06-28T20:08:20.685641+0000","last_active":"2023-06-28T20:08:22.108041+0000","last_peered":"2023-06-28T20:08:22.108041+0000","last_clean":"2023-06-28T20:08:22.108041+0000","last_became_active":"2023-06-28T20:08:20.685264+0000","last_became_peered":"2023-06-28T20:08:20.685264+0000","last_unstale":"2023-06-28T20:08:22.108041+0000","last_undegraded":"2023-06-28T20:08:22.108041+0000","last_fullsized":"2023-06-28T20:08:22.108041+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:58:25.349192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.658855+0000","last_change":"2023-06-28T20:08:20.682793+0000","last_active":"2023-06-28T20:08:21.658855+0000","last_peered":"2023-06-28T20:08:21.658855+0000","last_clean":"2023-06-28T20:08:21.658855+0000","last_became_active":"2023-06-28T20:08:20.682482+0000","last_became_peered":"2023-06-28T20:08:20.682482+0000","last_unstale":"2023-06-28T20:08:21.658855+0000","last_undegraded":"2023-06-28T20:08:21.658855+0000","last_fullsized":"2023-06-28T20:08:21.658855+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:58:15.735553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"34'4","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660257+0000","last_change":"2023-06-28T20:08:19.655460+0000","last_active":"2023-06-28T20:08:21.660257+0000","last_peered":"2023-06-28T20:08:21.660257+0000","last_clean":"2023-06-28T20:08:21.660257+0000","last_became_active":"2023-06-28T20:08:19.654505+0000","last_became_peered":"2023-06-28T20:08:19.654505+0000","last_unstale":"2023-06-28T20:08:21.660257+0000","last_undegraded":"2023-06-28T20:08:21.660257+0000","last_fullsized":"2023-06-28T20:08:21.660257+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:59:27.524440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109721+0000","last_change":"2023-06-28T20:08:20.680757+0000","last_active":"2023-06-28T20:08:22.109721+0000","last_peered":"2023-06-28T20:08:22.109721+0000","last_clean":"2023-06-28T20:08:22.109721+0000","last_became_active":"2023-06-28T20:08:20.680461+0000","last_became_peered":"2023-06-28T20:08:20.680461+0000","last_unstale":"2023-06-28T20:08:22.109721+0000","last_undegraded":"2023-06-28T20:08:22.109721+0000","last_fullsized":"2023-06-28T20:08:22.109721+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:15:25.870011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108368+0000","last_change":"2023-06-28T20:08:19.661131+0000","last_active":"2023-06-28T20:08:22.108368+0000","last_peered":"2023-06-28T20:08:22.108368+0000","last_clean":"2023-06-28T20:08:22.108368+0000","last_became_active":"2023-06-28T20:08:19.660874+0000","last_became_peered":"2023-06-28T20:08:19.660874+0000","last_unstale":"2023-06-28T20:08:22.108368+0000","last_undegraded":"2023-06-28T20:08:22.108368+0000","last_fullsized":"2023-06-28T20:08:22.108368+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:52:00.178605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659785+0000","last_change":"2023-06-28T20:08:20.665348+0000","last_active":"2023-06-28T20:08:21.659785+0000","last_peered":"2023-06-28T20:08:21.659785+0000","last_clean":"2023-06-28T20:08:21.659785+0000","last_became_active":"2023-06-28T20:08:20.664971+0000","last_became_peered":"2023-06-28T20:08:20.664971+0000","last_unstale":"2023-06-28T20:08:21.659785+0000","last_undegraded":"2023-06-28T20:08:21.659785+0000","last_fullsized":"2023-06-28T20:08:21.659785+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:20:18.739296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109782+0000","last_change":"2023-06-28T20:08:19.662313+0000","last_active":"2023-06-28T20:08:22.109782+0000","last_peered":"2023-06-28T20:08:22.109782+0000","last_clean":"2023-06-28T20:08:22.109782+0000","last_became_active":"2023-06-28T20:08:19.661987+0000","last_became_peered":"2023-06-28T20:08:19.661987+0000","last_unstale":"2023-06-28T20:08:22.109782+0000","last_undegraded":"2023-06-28T20:08:22.109782+0000","last_fullsized":"2023-06-28T20:08:22.109782+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:35:24.723033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109079+0000","last_change":"2023-06-28T20:08:20.675769+0000","last_active":"2023-06-28T20:08:22.109079+0000","last_peered":"2023-06-28T20:08:22.109079+0000","last_clean":"2023-06-28T20:08:22.109079+0000","last_became_active":"2023-06-28T20:08:20.675425+0000","last_became_peered":"2023-06-28T20:08:20.675425+0000","last_unstale":"2023-06-28T20:08:22.109079+0000","last_undegraded":"2023-06-28T20:08:22.109079+0000","last_fullsized":"2023-06-28T20:08:22.109079+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:57:32.895917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659585+0000","last_change":"2023-06-28T20:08:19.667027+0000","last_active":"2023-06-28T20:08:21.659585+0000","last_peered":"2023-06-28T20:08:21.659585+0000","last_clean":"2023-06-28T20:08:21.659585+0000","last_became_active":"2023-06-28T20:08:19.666747+0000","last_became_peered":"2023-06-28T20:08:19.666747+0000","last_unstale":"2023-06-28T20:08:21.659585+0000","last_undegraded":"2023-06-28T20:08:21.659585+0000","last_fullsized":"2023-06-28T20:08:21.659585+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:28:50.129574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108175+0000","last_change":"2023-06-28T20:08:20.677333+0000","last_active":"2023-06-28T20:08:22.108175+0000","last_peered":"2023-06-28T20:08:22.108175+0000","last_clean":"2023-06-28T20:08:22.108175+0000","last_became_active":"2023-06-28T20:08:20.677065+0000","last_became_peered":"2023-06-28T20:08:20.677065+0000","last_unstale":"2023-06-28T20:08:22.108175+0000","last_undegraded":"2023-06-28T20:08:22.108175+0000","last_fullsized":"2023-06-28T20:08:22.108175+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:11:02.786663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801292+0000","last_change":"2023-06-28T20:08:19.665791+0000","last_active":"2023-06-28T20:08:21.801292+0000","last_peered":"2023-06-28T20:08:21.801292+0000","last_clean":"2023-06-28T20:08:21.801292+0000","last_became_active":"2023-06-28T20:08:19.665360+0000","last_became_peered":"2023-06-28T20:08:19.665360+0000","last_unstale":"2023-06-28T20:08:21.801292+0000","last_undegraded":"2023-06-28T20:08:21.801292+0000","last_fullsized":"2023-06-28T20:08:21.801292+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:15:53.665554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801077+0000","last_change":"2023-06-28T20:08:20.684309+0000","last_active":"2023-06-28T20:08:21.801077+0000","last_peered":"2023-06-28T20:08:21.801077+0000","last_clean":"2023-06-28T20:08:21.801077+0000","last_became_active":"2023-06-28T20:08:20.684033+0000","last_became_peered":"2023-06-28T20:08:20.684033+0000","last_unstale":"2023-06-28T20:08:21.801077+0000","last_undegraded":"2023-06-28T20:08:21.801077+0000","last_fullsized":"2023-06-28T20:08:21.801077+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:35:25.727643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.658559+0000","last_change":"2023-06-28T20:08:19.663213+0000","last_active":"2023-06-28T20:08:21.658559+0000","last_peered":"2023-06-28T20:08:21.658559+0000","last_clean":"2023-06-28T20:08:21.658559+0000","last_became_active":"2023-06-28T20:08:19.662945+0000","last_became_peered":"2023-06-28T20:08:19.662945+0000","last_unstale":"2023-06-28T20:08:21.658559+0000","last_undegraded":"2023-06-28T20:08:21.658559+0000","last_fullsized":"2023-06-28T20:08:21.658559+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:00:05.808637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659321+0000","last_change":"2023-06-28T20:08:20.680144+0000","last_active":"2023-06-28T20:08:21.659321+0000","last_peered":"2023-06-28T20:08:21.659321+0000","last_clean":"2023-06-28T20:08:21.659321+0000","last_became_active":"2023-06-28T20:08:20.679612+0000","last_became_peered":"2023-06-28T20:08:20.679612+0000","last_unstale":"2023-06-28T20:08:21.659321+0000","last_undegraded":"2023-06-28T20:08:21.659321+0000","last_fullsized":"2023-06-28T20:08:21.659321+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:53:20.326887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109273+0000","last_change":"2023-06-28T20:08:19.664997+0000","last_active":"2023-06-28T20:08:22.109273+0000","last_peered":"2023-06-28T20:08:22.109273+0000","last_clean":"2023-06-28T20:08:22.109273+0000","last_became_active":"2023-06-28T20:08:19.664694+0000","last_became_peered":"2023-06-28T20:08:19.664694+0000","last_unstale":"2023-06-28T20:08:22.109273+0000","last_undegraded":"2023-06-28T20:08:22.109273+0000","last_fullsized":"2023-06-28T20:08:22.109273+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:05:45.969682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109030+0000","last_change":"2023-06-28T20:08:20.670512+0000","last_active":"2023-06-28T20:08:22.109030+0000","last_peered":"2023-06-28T20:08:22.109030+0000","last_clean":"2023-06-28T20:08:22.109030+0000","last_became_active":"2023-06-28T20:08:20.669897+0000","last_became_peered":"2023-06-28T20:08:20.669897+0000","last_unstale":"2023-06-28T20:08:22.109030+0000","last_undegraded":"2023-06-28T20:08:22.109030+0000","last_fullsized":"2023-06-28T20:08:22.109030+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:54:06.279406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659345+0000","last_change":"2023-06-28T20:08:19.652654+0000","last_active":"2023-06-28T20:08:21.659345+0000","last_peered":"2023-06-28T20:08:21.659345+0000","last_clean":"2023-06-28T20:08:21.659345+0000","last_became_active":"2023-06-28T20:08:19.652390+0000","last_became_peered":"2023-06-28T20:08:19.652390+0000","last_unstale":"2023-06-28T20:08:21.659345+0000","last_undegraded":"2023-06-28T20:08:21.659345+0000","last_fullsized":"2023-06-28T20:08:21.659345+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:03:20.512184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.798969+0000","last_change":"2023-06-28T20:08:20.685963+0000","last_active":"2023-06-28T20:08:21.798969+0000","last_peered":"2023-06-28T20:08:21.798969+0000","last_clean":"2023-06-28T20:08:21.798969+0000","last_became_active":"2023-06-28T20:08:20.685676+0000","last_became_peered":"2023-06-28T20:08:20.685676+0000","last_unstale":"2023-06-28T20:08:21.798969+0000","last_undegraded":"2023-06-28T20:08:21.798969+0000","last_fullsized":"2023-06-28T20:08:21.798969+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:58:50.702496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109250+0000","last_change":"2023-06-28T20:08:19.666008+0000","last_active":"2023-06-28T20:08:22.109250+0000","last_peered":"2023-06-28T20:08:22.109250+0000","last_clean":"2023-06-28T20:08:22.109250+0000","last_became_active":"2023-06-28T20:08:19.665730+0000","last_became_peered":"2023-06-28T20:08:19.665730+0000","last_unstale":"2023-06-28T20:08:22.109250+0000","last_undegraded":"2023-06-28T20:08:22.109250+0000","last_fullsized":"2023-06-28T20:08:22.109250+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:04:49.884359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660250+0000","last_change":"2023-06-28T20:08:20.667701+0000","last_active":"2023-06-28T20:08:21.660250+0000","last_peered":"2023-06-28T20:08:21.660250+0000","last_clean":"2023-06-28T20:08:21.660250+0000","last_became_active":"2023-06-28T20:08:20.667195+0000","last_became_peered":"2023-06-28T20:08:20.667195+0000","last_unstale":"2023-06-28T20:08:21.660250+0000","last_undegraded":"2023-06-28T20:08:21.660250+0000","last_fullsized":"2023-06-28T20:08:21.660250+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T00:01:31.303207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801689+0000","last_change":"2023-06-28T20:08:19.667830+0000","last_active":"2023-06-28T20:08:21.801689+0000","last_peered":"2023-06-28T20:08:21.801689+0000","last_clean":"2023-06-28T20:08:21.801689+0000","last_became_active":"2023-06-28T20:08:19.667590+0000","last_became_peered":"2023-06-28T20:08:19.667590+0000","last_unstale":"2023-06-28T20:08:21.801689+0000","last_undegraded":"2023-06-28T20:08:21.801689+0000","last_fullsized":"2023-06-28T20:08:21.801689+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:58:14.730943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109735+0000","last_change":"2023-06-28T20:08:20.676603+0000","last_active":"2023-06-28T20:08:22.109735+0000","last_peered":"2023-06-28T20:08:22.109735+0000","last_clean":"2023-06-28T20:08:22.109735+0000","last_became_active":"2023-06-28T20:08:20.676293+0000","last_became_peered":"2023-06-28T20:08:20.676293+0000","last_unstale":"2023-06-28T20:08:22.109735+0000","last_undegraded":"2023-06-28T20:08:22.109735+0000","last_fullsized":"2023-06-28T20:08:22.109735+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:04:56.887142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"34'3","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801684+0000","last_change":"2023-06-28T20:08:19.666930+0000","last_active":"2023-06-28T20:08:21.801684+0000","last_peered":"2023-06-28T20:08:21.801684+0000","last_clean":"2023-06-28T20:08:21.801684+0000","last_became_active":"2023-06-28T20:08:19.666639+0000","last_became_peered":"2023-06-28T20:08:19.666639+0000","last_unstale":"2023-06-28T20:08:21.801684+0000","last_undegraded":"2023-06-28T20:08:21.801684+0000","last_fullsized":"2023-06-28T20:08:21.801684+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:57:32.382031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659780+0000","last_change":"2023-06-28T20:08:20.685705+0000","last_active":"2023-06-28T20:08:21.659780+0000","last_peered":"2023-06-28T20:08:21.659780+0000","last_clean":"2023-06-28T20:08:21.659780+0000","last_became_active":"2023-06-28T20:08:20.685410+0000","last_became_peered":"2023-06-28T20:08:20.685410+0000","last_unstale":"2023-06-28T20:08:21.659780+0000","last_undegraded":"2023-06-28T20:08:21.659780+0000","last_fullsized":"2023-06-28T20:08:21.659780+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:16:41.819174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"34'6","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108711+0000","last_change":"2023-06-28T20:08:19.667034+0000","last_active":"2023-06-28T20:08:22.108711+0000","last_peered":"2023-06-28T20:08:22.108711+0000","last_clean":"2023-06-28T20:08:22.108711+0000","last_became_active":"2023-06-28T20:08:19.666621+0000","last_became_peered":"2023-06-28T20:08:19.666621+0000","last_unstale":"2023-06-28T20:08:22.108711+0000","last_undegraded":"2023-06-28T20:08:22.108711+0000","last_fullsized":"2023-06-28T20:08:22.108711+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:13:48.941368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109853+0000","last_change":"2023-06-28T20:08:20.681551+0000","last_active":"2023-06-28T20:08:22.109853+0000","last_peered":"2023-06-28T20:08:22.109853+0000","last_clean":"2023-06-28T20:08:22.109853+0000","last_became_active":"2023-06-28T20:08:20.681224+0000","last_became_peered":"2023-06-28T20:08:20.681224+0000","last_unstale":"2023-06-28T20:08:22.109853+0000","last_undegraded":"2023-06-28T20:08:22.109853+0000","last_fullsized":"2023-06-28T20:08:22.109853+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:52:36.907231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109088+0000","last_change":"2023-06-28T20:08:19.664339+0000","last_active":"2023-06-28T20:08:22.109088+0000","last_peered":"2023-06-28T20:08:22.109088+0000","last_clean":"2023-06-28T20:08:22.109088+0000","last_became_active":"2023-06-28T20:08:19.664043+0000","last_became_peered":"2023-06-28T20:08:19.664043+0000","last_unstale":"2023-06-28T20:08:22.109088+0000","last_undegraded":"2023-06-28T20:08:22.109088+0000","last_fullsized":"2023-06-28T20:08:22.109088+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:03:23.231723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108943+0000","last_change":"2023-06-28T20:08:20.675759+0000","last_active":"2023-06-28T20:08:22.108943+0000","last_peered":"2023-06-28T20:08:22.108943+0000","last_clean":"2023-06-28T20:08:22.108943+0000","last_became_active":"2023-06-28T20:08:20.675395+0000","last_became_peered":"2023-06-28T20:08:20.675395+0000","last_unstale":"2023-06-28T20:08:22.108943+0000","last_undegraded":"2023-06-28T20:08:22.108943+0000","last_fullsized":"2023-06-28T20:08:22.108943+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:00:56.177481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109633+0000","last_change":"2023-06-28T20:08:19.667168+0000","last_active":"2023-06-28T20:08:22.109633+0000","last_peered":"2023-06-28T20:08:22.109633+0000","last_clean":"2023-06-28T20:08:22.109633+0000","last_became_active":"2023-06-28T20:08:19.666902+0000","last_became_peered":"2023-06-28T20:08:19.666902+0000","last_unstale":"2023-06-28T20:08:22.109633+0000","last_undegraded":"2023-06-28T20:08:22.109633+0000","last_fullsized":"2023-06-28T20:08:22.109633+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:04:55.882533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.658311+0000","last_change":"2023-06-28T20:08:20.684341+0000","last_active":"2023-06-28T20:08:21.658311+0000","last_peered":"2023-06-28T20:08:21.658311+0000","last_clean":"2023-06-28T20:08:21.658311+0000","last_became_active":"2023-06-28T20:08:20.684028+0000","last_became_peered":"2023-06-28T20:08:20.684028+0000","last_unstale":"2023-06-28T20:08:21.658311+0000","last_undegraded":"2023-06-28T20:08:21.658311+0000","last_fullsized":"2023-06-28T20:08:21.658311+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:15:54.670164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659744+0000","last_change":"2023-06-28T20:08:19.653185+0000","last_active":"2023-06-28T20:08:21.659744+0000","last_peered":"2023-06-28T20:08:21.659744+0000","last_clean":"2023-06-28T20:08:21.659744+0000","last_became_active":"2023-06-28T20:08:19.652574+0000","last_became_peered":"2023-06-28T20:08:19.652574+0000","last_unstale":"2023-06-28T20:08:21.659744+0000","last_undegraded":"2023-06-28T20:08:21.659744+0000","last_fullsized":"2023-06-28T20:08:21.659744+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:59:46.241857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108883+0000","last_change":"2023-06-28T20:08:20.679542+0000","last_active":"2023-06-28T20:08:22.108883+0000","last_peered":"2023-06-28T20:08:22.108883+0000","last_clean":"2023-06-28T20:08:22.108883+0000","last_became_active":"2023-06-28T20:08:20.679243+0000","last_became_peered":"2023-06-28T20:08:20.679243+0000","last_unstale":"2023-06-28T20:08:22.108883+0000","last_undegraded":"2023-06-28T20:08:22.108883+0000","last_fullsized":"2023-06-28T20:08:22.108883+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:39:58.915632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"34'8","reported_seq":21,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659344+0000","last_change":"2023-06-28T20:08:19.663890+0000","last_active":"2023-06-28T20:08:21.659344+0000","last_peered":"2023-06-28T20:08:21.659344+0000","last_clean":"2023-06-28T20:08:21.659344+0000","last_became_active":"2023-06-28T20:08:19.663624+0000","last_became_peered":"2023-06-28T20:08:19.663624+0000","last_unstale":"2023-06-28T20:08:21.659344+0000","last_undegraded":"2023-06-28T20:08:21.659344+0000","last_fullsized":"2023-06-28T20:08:21.659344+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:22:55.905699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108621+0000","last_change":"2023-06-28T20:08:20.670804+0000","last_active":"2023-06-28T20:08:22.108621+0000","last_peered":"2023-06-28T20:08:22.108621+0000","last_clean":"2023-06-28T20:08:22.108621+0000","last_became_active":"2023-06-28T20:08:20.670467+0000","last_became_peered":"2023-06-28T20:08:20.670467+0000","last_unstale":"2023-06-28T20:08:22.108621+0000","last_undegraded":"2023-06-28T20:08:22.108621+0000","last_fullsized":"2023-06-28T20:08:22.108621+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:45:08.309241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801739+0000","last_change":"2023-06-28T20:08:19.663175+0000","last_active":"2023-06-28T20:08:21.801739+0000","last_peered":"2023-06-28T20:08:21.801739+0000","last_clean":"2023-06-28T20:08:21.801739+0000","last_became_active":"2023-06-28T20:08:19.662835+0000","last_became_peered":"2023-06-28T20:08:19.662835+0000","last_unstale":"2023-06-28T20:08:21.801739+0000","last_undegraded":"2023-06-28T20:08:21.801739+0000","last_fullsized":"2023-06-28T20:08:21.801739+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:42:20.628345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801527+0000","last_change":"2023-06-28T20:08:20.684077+0000","last_active":"2023-06-28T20:08:21.801527+0000","last_peered":"2023-06-28T20:08:21.801527+0000","last_clean":"2023-06-28T20:08:21.801527+0000","last_became_active":"2023-06-28T20:08:20.683782+0000","last_became_peered":"2023-06-28T20:08:20.683782+0000","last_unstale":"2023-06-28T20:08:21.801527+0000","last_undegraded":"2023-06-28T20:08:21.801527+0000","last_fullsized":"2023-06-28T20:08:21.801527+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:19:46.143213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659977+0000","last_change":"2023-06-28T20:08:19.664515+0000","last_active":"2023-06-28T20:08:21.659977+0000","last_peered":"2023-06-28T20:08:21.659977+0000","last_clean":"2023-06-28T20:08:21.659977+0000","last_became_active":"2023-06-28T20:08:19.663664+0000","last_became_peered":"2023-06-28T20:08:19.663664+0000","last_unstale":"2023-06-28T20:08:21.659977+0000","last_undegraded":"2023-06-28T20:08:21.659977+0000","last_fullsized":"2023-06-28T20:08:21.659977+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:31:54.171322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659901+0000","last_change":"2023-06-28T20:08:20.667446+0000","last_active":"2023-06-28T20:08:21.659901+0000","last_peered":"2023-06-28T20:08:21.659901+0000","last_clean":"2023-06-28T20:08:21.659901+0000","last_became_active":"2023-06-28T20:08:20.667193+0000","last_became_peered":"2023-06-28T20:08:20.667193+0000","last_unstale":"2023-06-28T20:08:21.659901+0000","last_undegraded":"2023-06-28T20:08:21.659901+0000","last_fullsized":"2023-06-28T20:08:21.659901+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:13:52.178823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"34'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.799066+0000","last_change":"2023-06-28T20:08:19.662299+0000","last_active":"2023-06-28T20:08:21.799066+0000","last_peered":"2023-06-28T20:08:21.799066+0000","last_clean":"2023-06-28T20:08:21.799066+0000","last_became_active":"2023-06-28T20:08:19.662002+0000","last_became_peered":"2023-06-28T20:08:19.662002+0000","last_unstale":"2023-06-28T20:08:21.799066+0000","last_undegraded":"2023-06-28T20:08:21.799066+0000","last_fullsized":"2023-06-28T20:08:21.799066+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:16:57.488453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.798999+0000","last_change":"2023-06-28T20:08:20.688515+0000","last_active":"2023-06-28T20:08:21.798999+0000","last_peered":"2023-06-28T20:08:21.798999+0000","last_clean":"2023-06-28T20:08:21.798999+0000","last_became_active":"2023-06-28T20:08:20.688241+0000","last_became_peered":"2023-06-28T20:08:20.688241+0000","last_unstale":"2023-06-28T20:08:21.798999+0000","last_undegraded":"2023-06-28T20:08:21.798999+0000","last_fullsized":"2023-06-28T20:08:21.798999+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:48:31.571168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109592+0000","last_change":"2023-06-28T20:08:19.661197+0000","last_active":"2023-06-28T20:08:22.109592+0000","last_peered":"2023-06-28T20:08:22.109592+0000","last_clean":"2023-06-28T20:08:22.109592+0000","last_became_active":"2023-06-28T20:08:19.660788+0000","last_became_peered":"2023-06-28T20:08:19.660788+0000","last_unstale":"2023-06-28T20:08:22.109592+0000","last_undegraded":"2023-06-28T20:08:22.109592+0000","last_fullsized":"2023-06-28T20:08:22.109592+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:39:24.497671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660440+0000","last_change":"2023-06-28T20:08:20.686973+0000","last_active":"2023-06-28T20:08:21.660440+0000","last_peered":"2023-06-28T20:08:21.660440+0000","last_clean":"2023-06-28T20:08:21.660440+0000","last_became_active":"2023-06-28T20:08:20.686719+0000","last_became_peered":"2023-06-28T20:08:20.686719+0000","last_unstale":"2023-06-28T20:08:21.660440+0000","last_undegraded":"2023-06-28T20:08:21.660440+0000","last_fullsized":"2023-06-28T20:08:21.660440+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:57:33.386641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.798859+0000","last_change":"2023-06-28T20:08:19.653505+0000","last_active":"2023-06-28T20:08:21.798859+0000","last_peered":"2023-06-28T20:08:21.798859+0000","last_clean":"2023-06-28T20:08:21.798859+0000","last_became_active":"2023-06-28T20:08:19.653079+0000","last_became_peered":"2023-06-28T20:08:19.653079+0000","last_unstale":"2023-06-28T20:08:21.798859+0000","last_undegraded":"2023-06-28T20:08:21.798859+0000","last_fullsized":"2023-06-28T20:08:21.798859+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:18:23.057931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109808+0000","last_change":"2023-06-28T20:08:20.675012+0000","last_active":"2023-06-28T20:08:22.109808+0000","last_peered":"2023-06-28T20:08:22.109808+0000","last_clean":"2023-06-28T20:08:22.109808+0000","last_became_active":"2023-06-28T20:08:20.674703+0000","last_became_peered":"2023-06-28T20:08:20.674703+0000","last_unstale":"2023-06-28T20:08:22.109808+0000","last_undegraded":"2023-06-28T20:08:22.109808+0000","last_fullsized":"2023-06-28T20:08:22.109808+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:35:25.727643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.798741+0000","last_change":"2023-06-28T20:08:19.652706+0000","last_active":"2023-06-28T20:08:21.798741+0000","last_peered":"2023-06-28T20:08:21.798741+0000","last_clean":"2023-06-28T20:08:21.798741+0000","last_became_active":"2023-06-28T20:08:19.652359+0000","last_became_peered":"2023-06-28T20:08:19.652359+0000","last_unstale":"2023-06-28T20:08:21.798741+0000","last_undegraded":"2023-06-28T20:08:21.798741+0000","last_fullsized":"2023-06-28T20:08:21.798741+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:18:28.900535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801156+0000","last_change":"2023-06-28T20:08:20.679109+0000","last_active":"2023-06-28T20:08:21.801156+0000","last_peered":"2023-06-28T20:08:21.801156+0000","last_clean":"2023-06-28T20:08:21.801156+0000","last_became_active":"2023-06-28T20:08:20.678833+0000","last_became_peered":"2023-06-28T20:08:20.678833+0000","last_unstale":"2023-06-28T20:08:21.801156+0000","last_undegraded":"2023-06-28T20:08:21.801156+0000","last_fullsized":"2023-06-28T20:08:21.801156+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:25:08.187049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.658489+0000","last_change":"2023-06-28T20:08:19.664718+0000","last_active":"2023-06-28T20:08:21.658489+0000","last_peered":"2023-06-28T20:08:21.658489+0000","last_clean":"2023-06-28T20:08:21.658489+0000","last_became_active":"2023-06-28T20:08:19.664456+0000","last_became_peered":"2023-06-28T20:08:19.664456+0000","last_unstale":"2023-06-28T20:08:21.658489+0000","last_undegraded":"2023-06-28T20:08:21.658489+0000","last_fullsized":"2023-06-28T20:08:21.658489+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:59:27.524440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108945+0000","last_change":"2023-06-28T20:08:20.670715+0000","last_active":"2023-06-28T20:08:22.108945+0000","last_peered":"2023-06-28T20:08:22.108945+0000","last_clean":"2023-06-28T20:08:22.108945+0000","last_became_active":"2023-06-28T20:08:20.670255+0000","last_became_peered":"2023-06-28T20:08:20.670255+0000","last_unstale":"2023-06-28T20:08:22.108945+0000","last_undegraded":"2023-06-28T20:08:22.108945+0000","last_fullsized":"2023-06-28T20:08:22.108945+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:45:29.637108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.798735+0000","last_change":"2023-06-28T20:08:19.659806+0000","last_active":"2023-06-28T20:08:21.798735+0000","last_peered":"2023-06-28T20:08:21.798735+0000","last_clean":"2023-06-28T20:08:21.798735+0000","last_became_active":"2023-06-28T20:08:19.659485+0000","last_became_peered":"2023-06-28T20:08:19.659485+0000","last_unstale":"2023-06-28T20:08:21.798735+0000","last_undegraded":"2023-06-28T20:08:21.798735+0000","last_fullsized":"2023-06-28T20:08:21.798735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:26:26.268929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659905+0000","last_change":"2023-06-28T20:08:20.673389+0000","last_active":"2023-06-28T20:08:21.659905+0000","last_peered":"2023-06-28T20:08:21.659905+0000","last_clean":"2023-06-28T20:08:21.659905+0000","last_became_active":"2023-06-28T20:08:20.672447+0000","last_became_peered":"2023-06-28T20:08:20.672447+0000","last_unstale":"2023-06-28T20:08:21.659905+0000","last_undegraded":"2023-06-28T20:08:21.659905+0000","last_fullsized":"2023-06-28T20:08:21.659905+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:31:45.241155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659003+0000","last_change":"2023-06-28T20:08:19.665324+0000","last_active":"2023-06-28T20:08:21.659003+0000","last_peered":"2023-06-28T20:08:21.659003+0000","last_clean":"2023-06-28T20:08:21.659003+0000","last_became_active":"2023-06-28T20:08:19.664984+0000","last_became_peered":"2023-06-28T20:08:19.664984+0000","last_unstale":"2023-06-28T20:08:21.659003+0000","last_undegraded":"2023-06-28T20:08:21.659003+0000","last_fullsized":"2023-06-28T20:08:21.659003+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:19:03.246307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":37,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659501+0000","last_change":"2023-06-28T20:07:15.815425+0000","last_active":"2023-06-28T20:08:21.659501+0000","last_peered":"2023-06-28T20:08:21.659501+0000","last_clean":"2023-06-28T20:08:21.659501+0000","last_became_active":"2023-06-28T20:07:15.815163+0000","last_became_peered":"2023-06-28T20:07:15.815163+0000","last_unstale":"2023-06-28T20:08:21.659501+0000","last_undegraded":"2023-06-28T20:08:21.659501+0000","last_fullsized":"2023-06-28T20:08:21.659501+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:39:02.649739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.658477+0000","last_change":"2023-06-28T20:08:20.664815+0000","last_active":"2023-06-28T20:08:21.658477+0000","last_peered":"2023-06-28T20:08:21.658477+0000","last_clean":"2023-06-28T20:08:21.658477+0000","last_became_active":"2023-06-28T20:08:20.664482+0000","last_became_peered":"2023-06-28T20:08:20.664482+0000","last_unstale":"2023-06-28T20:08:21.658477+0000","last_undegraded":"2023-06-28T20:08:21.658477+0000","last_fullsized":"2023-06-28T20:08:21.658477+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:19:46.143213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108680+0000","last_change":"2023-06-28T20:08:19.663072+0000","last_active":"2023-06-28T20:08:22.108680+0000","last_peered":"2023-06-28T20:08:22.108680+0000","last_clean":"2023-06-28T20:08:22.108680+0000","last_became_active":"2023-06-28T20:08:19.662730+0000","last_became_peered":"2023-06-28T20:08:19.662730+0000","last_unstale":"2023-06-28T20:08:22.108680+0000","last_undegraded":"2023-06-28T20:08:22.108680+0000","last_fullsized":"2023-06-28T20:08:22.108680+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:02:45.086997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":43,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108762+0000","last_change":"2023-06-28T20:07:15.303675+0000","last_active":"2023-06-28T20:08:22.108762+0000","last_peered":"2023-06-28T20:08:22.108762+0000","last_clean":"2023-06-28T20:08:22.108762+0000","last_became_active":"2023-06-28T20:07:15.303350+0000","last_became_peered":"2023-06-28T20:07:15.303350+0000","last_unstale":"2023-06-28T20:08:22.108762+0000","last_undegraded":"2023-06-28T20:08:22.108762+0000","last_fullsized":"2023-06-28T20:08:22.108762+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:12:06.137754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109443+0000","last_change":"2023-06-28T20:08:20.682863+0000","last_active":"2023-06-28T20:08:22.109443+0000","last_peered":"2023-06-28T20:08:22.109443+0000","last_clean":"2023-06-28T20:08:22.109443+0000","last_became_active":"2023-06-28T20:08:20.682499+0000","last_became_peered":"2023-06-28T20:08:20.682499+0000","last_unstale":"2023-06-28T20:08:22.109443+0000","last_undegraded":"2023-06-28T20:08:22.109443+0000","last_fullsized":"2023-06-28T20:08:22.109443+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:56:18.035434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"34'2","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801805+0000","last_change":"2023-06-28T20:08:19.666978+0000","last_active":"2023-06-28T20:08:21.801805+0000","last_peered":"2023-06-28T20:08:21.801805+0000","last_clean":"2023-06-28T20:08:21.801805+0000","last_became_active":"2023-06-28T20:08:19.666613+0000","last_became_peered":"2023-06-28T20:08:19.666613+0000","last_unstale":"2023-06-28T20:08:21.801805+0000","last_undegraded":"2023-06-28T20:08:21.801805+0000","last_fullsized":"2023-06-28T20:08:21.801805+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:22:55.905699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":37,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109348+0000","last_change":"2023-06-28T20:07:15.303565+0000","last_active":"2023-06-28T20:08:22.109348+0000","last_peered":"2023-06-28T20:08:22.109348+0000","last_clean":"2023-06-28T20:08:22.109348+0000","last_became_active":"2023-06-28T20:07:15.303200+0000","last_became_peered":"2023-06-28T20:07:15.303200+0000","last_unstale":"2023-06-28T20:08:22.109348+0000","last_undegraded":"2023-06-28T20:08:22.109348+0000","last_fullsized":"2023-06-28T20:08:22.109348+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:58:08.218147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":185,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659552+0000","last_change":"2023-06-28T20:07:15.304921+0000","last_active":"2023-06-28T20:08:21.659552+0000","last_peered":"2023-06-28T20:08:21.659552+0000","last_clean":"2023-06-28T20:08:21.659552+0000","last_became_active":"2023-06-28T20:07:15.303959+0000","last_became_peered":"2023-06-28T20:07:15.303959+0000","last_unstale":"2023-06-28T20:08:21.659552+0000","last_undegraded":"2023-06-28T20:08:21.659552+0000","last_fullsized":"2023-06-28T20:08:21.659552+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:21:11.722702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660392+0000","last_change":"2023-06-28T20:08:20.667481+0000","last_active":"2023-06-28T20:08:21.660392+0000","last_peered":"2023-06-28T20:08:21.660392+0000","last_clean":"2023-06-28T20:08:21.660392+0000","last_became_active":"2023-06-28T20:08:20.666850+0000","last_became_peered":"2023-06-28T20:08:20.666850+0000","last_unstale":"2023-06-28T20:08:21.660392+0000","last_undegraded":"2023-06-28T20:08:21.660392+0000","last_fullsized":"2023-06-28T20:08:21.660392+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:37:01.278070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"35'1","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.800649+0000","last_change":"2023-06-28T20:08:19.664561+0000","last_active":"2023-06-28T20:08:21.800649+0000","last_peered":"2023-06-28T20:08:21.800649+0000","last_clean":"2023-06-28T20:08:21.800649+0000","last_became_active":"2023-06-28T20:08:19.663684+0000","last_became_peered":"2023-06-28T20:08:19.663684+0000","last_unstale":"2023-06-28T20:08:21.800649+0000","last_undegraded":"2023-06-28T20:08:21.800649+0000","last_fullsized":"2023-06-28T20:08:21.800649+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:33:29.127873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":37,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.658346+0000","last_change":"2023-06-28T20:07:15.899048+0000","last_active":"2023-06-28T20:08:21.658346+0000","last_peered":"2023-06-28T20:08:21.658346+0000","last_clean":"2023-06-28T20:08:21.658346+0000","last_became_active":"2023-06-28T20:07:15.898766+0000","last_became_peered":"2023-06-28T20:07:15.898766+0000","last_unstale":"2023-06-28T20:08:21.658346+0000","last_undegraded":"2023-06-28T20:08:21.658346+0000","last_fullsized":"2023-06-28T20:08:21.658346+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:04:24.646683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801713+0000","last_change":"2023-06-28T20:08:20.680186+0000","last_active":"2023-06-28T20:08:21.801713+0000","last_peered":"2023-06-28T20:08:21.801713+0000","last_clean":"2023-06-28T20:08:21.801713+0000","last_became_active":"2023-06-28T20:08:20.679936+0000","last_became_peered":"2023-06-28T20:08:20.679936+0000","last_unstale":"2023-06-28T20:08:21.801713+0000","last_undegraded":"2023-06-28T20:08:21.801713+0000","last_fullsized":"2023-06-28T20:08:21.801713+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:20:23.086774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.658977+0000","last_change":"2023-06-28T20:08:19.662452+0000","last_active":"2023-06-28T20:08:21.658977+0000","last_peered":"2023-06-28T20:08:21.658977+0000","last_clean":"2023-06-28T20:08:21.658977+0000","last_became_active":"2023-06-28T20:08:19.661640+0000","last_became_peered":"2023-06-28T20:08:19.661640+0000","last_unstale":"2023-06-28T20:08:21.658977+0000","last_undegraded":"2023-06-28T20:08:21.658977+0000","last_fullsized":"2023-06-28T20:08:21.658977+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:20:22.082165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":35,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.659431+0000","last_change":"2023-06-28T20:07:15.899229+0000","last_active":"2023-06-28T20:08:21.659431+0000","last_peered":"2023-06-28T20:08:21.659431+0000","last_clean":"2023-06-28T20:08:21.659431+0000","last_became_active":"2023-06-28T20:07:15.898708+0000","last_became_peered":"2023-06-28T20:07:15.898708+0000","last_unstale":"2023-06-28T20:08:21.659431+0000","last_undegraded":"2023-06-28T20:08:21.659431+0000","last_fullsized":"2023-06-28T20:08:21.659431+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:48:33.012020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108808+0000","last_change":"2023-06-28T20:08:20.674909+0000","last_active":"2023-06-28T20:08:22.108808+0000","last_peered":"2023-06-28T20:08:22.108808+0000","last_clean":"2023-06-28T20:08:22.108808+0000","last_became_active":"2023-06-28T20:08:20.671930+0000","last_became_peered":"2023-06-28T20:08:20.671930+0000","last_unstale":"2023-06-28T20:08:22.108808+0000","last_undegraded":"2023-06-28T20:08:22.108808+0000","last_fullsized":"2023-06-28T20:08:22.108808+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:58:40.898791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109245+0000","last_change":"2023-06-28T20:08:19.663982+0000","last_active":"2023-06-28T20:08:22.109245+0000","last_peered":"2023-06-28T20:08:22.109245+0000","last_clean":"2023-06-28T20:08:22.109245+0000","last_became_active":"2023-06-28T20:08:19.663657+0000","last_became_peered":"2023-06-28T20:08:19.663657+0000","last_unstale":"2023-06-28T20:08:22.109245+0000","last_undegraded":"2023-06-28T20:08:22.109245+0000","last_fullsized":"2023-06-28T20:08:22.109245+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:39:24.497671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.801240+0000","last_change":"2023-06-28T20:07:15.898966+0000","last_active":"2023-06-28T20:08:21.801240+0000","last_peered":"2023-06-28T20:08:21.801240+0000","last_clean":"2023-06-28T20:08:21.801240+0000","last_became_active":"2023-06-28T20:07:15.898704+0000","last_became_peered":"2023-06-28T20:07:15.898704+0000","last_unstale":"2023-06-28T20:08:21.801240+0000","last_undegraded":"2023-06-28T20:08:21.801240+0000","last_fullsized":"2023-06-28T20:08:21.801240+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T07:58:41.892167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108131+0000","last_change":"2023-06-28T20:08:20.667824+0000","last_active":"2023-06-28T20:08:22.108131+0000","last_peered":"2023-06-28T20:08:22.108131+0000","last_clean":"2023-06-28T20:08:22.108131+0000","last_became_active":"2023-06-28T20:08:20.667548+0000","last_became_peered":"2023-06-28T20:08:20.667548+0000","last_unstale":"2023-06-28T20:08:22.108131+0000","last_undegraded":"2023-06-28T20:08:22.108131+0000","last_fullsized":"2023-06-28T20:08:22.108131+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:11:02.786663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660615+0000","last_change":"2023-06-28T20:08:19.653324+0000","last_active":"2023-06-28T20:08:21.660615+0000","last_peered":"2023-06-28T20:08:21.660615+0000","last_clean":"2023-06-28T20:08:21.660615+0000","last_became_active":"2023-06-28T20:08:19.652941+0000","last_became_peered":"2023-06-28T20:08:19.652941+0000","last_unstale":"2023-06-28T20:08:21.660615+0000","last_undegraded":"2023-06-28T20:08:21.660615+0000","last_fullsized":"2023-06-28T20:08:21.660615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:20:22.082165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":47,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660032+0000","last_change":"2023-06-28T20:07:15.899207+0000","last_active":"2023-06-28T20:08:21.660032+0000","last_peered":"2023-06-28T20:08:21.660032+0000","last_clean":"2023-06-28T20:08:21.660032+0000","last_became_active":"2023-06-28T20:07:15.898908+0000","last_became_peered":"2023-06-28T20:07:15.898908+0000","last_unstale":"2023-06-28T20:08:21.660032+0000","last_undegraded":"2023-06-28T20:08:21.660032+0000","last_fullsized":"2023-06-28T20:08:21.660032+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:57:10.381253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.109908+0000","last_change":"2023-06-28T20:08:20.687344+0000","last_active":"2023-06-28T20:08:22.109908+0000","last_peered":"2023-06-28T20:08:22.109908+0000","last_clean":"2023-06-28T20:08:22.109908+0000","last_became_active":"2023-06-28T20:08:20.687061+0000","last_became_peered":"2023-06-28T20:08:20.687061+0000","last_unstale":"2023-06-28T20:08:22.109908+0000","last_undegraded":"2023-06-28T20:08:22.109908+0000","last_fullsized":"2023-06-28T20:08:22.109908+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:19.641333+0000","last_clean_scrub_stamp":"2023-06-28T20:08:19.641333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:15:25.870011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:22.108612+0000","last_change":"2023-06-28T20:08:19.650567+0000","last_active":"2023-06-28T20:08:22.108612+0000","last_peered":"2023-06-28T20:08:22.108612+0000","last_clean":"2023-06-28T20:08:22.108612+0000","last_became_active":"2023-06-28T20:08:19.650333+0000","last_became_peered":"2023-06-28T20:08:19.650333+0000","last_unstale":"2023-06-28T20:08:22.108612+0000","last_undegraded":"2023-06-28T20:08:22.108612+0000","last_fullsized":"2023-06-28T20:08:22.108612+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:18.636724+0000","last_clean_scrub_stamp":"2023-06-28T20:08:18.636724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:56:17.030824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":47,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-06-28T20:08:21.660200+0000","last_change":"2023-06-28T20:07:15.303873+0000","last_active":"2023-06-28T20:08:21.660200+0000","last_peered":"2023-06-28T20:08:21.660200+0000","last_clean":"2023-06-28T20:08:21.660200+0000","last_became_active":"2023-06-28T20:07:15.303457+0000","last_became_peered":"2023-06-28T20:07:15.303457+0000","last_unstale":"2023-06-28T20:08:21.660200+0000","last_undegraded":"2023-06-28T20:08:21.660200+0000","last_fullsized":"2023-06-28T20:08:21.660200+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:27:15.184713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":1007616,"data_stored":756976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.75},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.71099999999999997}]},{"osd":1,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.59499999999999997}]},{"osd":2,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.67100000000000004}]},{"osd":3,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.51600000000000001}]},{"osd":4,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.46200000000000002}]},{"osd":5,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.434},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.54900000000000004}]},{"osd":6,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.52900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":401408,"data_stored":165556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":9.0069999999999997,"5min":9.0069999999999997,"15min":9.0069999999999997},"last":0.58799999999999997}]},{"osd":1,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":3.036,"5min":3.036,"15min":3.036},"last":0.625},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.60899999999999999}]},{"osd":2,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.66900000000000004}]},{"osd":3,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":9.0459999999999994,"5min":9.0459999999999994,"15min":9.0459999999999994},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.69799999999999995}]},{"osd":4,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.42299999999999999}]},{"osd":5,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.51200000000000001}]},{"osd":7,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.53200000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":995328,"data_stored":755432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.26500000000000001}]},{"osd":2,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.34699999999999998}]},{"osd":3,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.36799999999999999}]},{"osd":4,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56599999999999995}]},{"osd":5,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59599999999999997}]},{"osd":6,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.497},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.624}]},{"osd":7,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":388,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":397312,"data_stored":164047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.219}]},{"osd":2,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.442,"5min":0.442,"15min":0.442},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.26600000000000001}]},{"osd":3,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.28499999999999998}]},{"osd":4,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.434},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.38700000000000001}]},{"osd":5,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.39400000000000002}]},{"osd":6,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.54500000000000004}]},{"osd":7,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.47199999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":401408,"data_stored":165044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":16.044,"5min":16.044,"15min":16.044},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.34499999999999997}]},{"osd":1,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":11.028,"5min":11.028,"15min":11.028},"last":0.27700000000000002},{"interface":"front","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":16.077000000000002,"5min":16.077000000000002,"15min":16.077000000000002},"last":0.42599999999999999}]},{"osd":3,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.36499999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.319,"15min":1.319},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":16.004000000000001,"5min":16.004000000000001,"15min":16.004000000000001},"last":0.38100000000000001}]},{"osd":4,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":9.327,"5min":9.327,"15min":9.327},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":10.964,"5min":10.964,"15min":10.964},"last":0.86499999999999999}]},{"osd":5,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":10.923,"5min":10.923,"15min":10.923},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.81699999999999995}]},{"osd":6,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":9.282,"5min":9.282,"15min":9.282},"last":0.59099999999999997}]},{"osd":7,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":10.996,"5min":10.996,"15min":10.996},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.76700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":413696,"data_stored":166572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":5.0250000000000004,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":0.46600000000000003}]},{"osd":1,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":5.2309999999999999,"5min":5.2309999999999999,"15min":5.2309999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":5.3250000000000002,"5min":5.3250000000000002,"15min":5.3250000000000002},"last":0.40100000000000002}]},{"osd":2,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":15.927,"5min":15.927,"15min":15.927},"last":0.44},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.36299999999999999}]},{"osd":4,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.5},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.52500000000000002}]},{"osd":5,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":4.9939999999999998,"5min":4.9939999999999998,"15min":4.9939999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.65400000000000003}]},{"osd":6,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":4.9619999999999997,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.59499999999999997}]},{"osd":7,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":4.9210000000000003,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.626}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":405504,"data_stored":165578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":9.1809999999999992,"5min":9.1809999999999992,"15min":9.1809999999999992},"last":0.39900000000000002}]},{"osd":1,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.627}]},{"osd":2,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":9.0700000000000003,"5min":9.0700000000000003,"15min":9.0700000000000003},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.64400000000000002}]},{"osd":3,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.60499999999999998}]},{"osd":5,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.753}]},{"osd":6,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.73599999999999999}]},{"osd":7,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.77200000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":405504,"data_stored":165086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.45200000000000001}]},{"osd":1,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.47799999999999998}]},{"osd":2,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.60799999999999998}]},{"osd":3,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":5.6139999999999999,"5min":5.6139999999999999,"15min":5.6139999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":5.6500000000000004,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.52600000000000002}]},{"osd":4,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.54800000000000004}]},{"osd":6,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.5},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.69799999999999995}]},{"osd":7,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.63600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-28T20:08:25.531 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-06-28T20:08:25.532 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-06-28T20:08:25.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.692+0000 7f46fb48a700 1 Processor -- start 2023-06-28T20:08:25.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.692+0000 7f46fb48a700 1 -- start start 2023-06-28T20:08:25.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.693+0000 7f46fb48a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46f40aec90 0x7f46f4160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:25.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.693+0000 7f46fb48a700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f46f406ccc0 0x7f46f40b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:25.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.693+0000 7f46fb48a700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46f4160250 con 0x7f46f40af060 2023-06-28T20:08:25.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.693+0000 7f46fb48a700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46f415cb30 con 0x7f46f406ccc0 2023-06-28T20:08:25.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.693+0000 7f46fb48a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f4071410 con 0x7f46f40aec90 2023-06-28T20:08:25.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.693+0000 7f46f8a25700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f46f40af060 0x7f46f4070bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44254/0 (socket says 172.21.15.43:44254) 2023-06-28T20:08:25.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.693+0000 7f46f8a25700 1 -- 172.21.15.43:0/498737200 learned_addr learned my addr 172.21.15.43:0/498737200 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:25.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.694+0000 7f46f9a27700 1 --2- 172.21.15.43:0/498737200 >> v2:172.21.15.178:3300/0 conn(0x7f46f406ccc0 0x7f46f40b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:25.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.694+0000 7f46ebfff700 1 -- 172.21.15.43:0/498737200 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1967594334 0 0) 0x7f46f4160250 con 0x7f46f40af060 2023-06-28T20:08:25.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.694+0000 7f46f9226700 1 --2- 172.21.15.43:0/498737200 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46f40aec90 0x7f46f4160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:25.701 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.694+0000 7f46ebfff700 1 -- 172.21.15.43:0/498737200 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46e00035b0 con 0x7f46f40af060 2023-06-28T20:08:25.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.694+0000 7f46ebfff700 1 -- 172.21.15.43:0/498737200 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4014589750 0 0) 0x7f46e00035b0 con 0x7f46f40af060 2023-06-28T20:08:25.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.694+0000 7f46ebfff700 1 -- 172.21.15.43:0/498737200 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46f40aec90 msgr2=0x7f46f4160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:25.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.694+0000 7f46ebfff700 1 --2- 172.21.15.43:0/498737200 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46f40aec90 0x7f46f4160cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.694+0000 7f46ebfff700 1 -- 172.21.15.43:0/498737200 >> v2:172.21.15.178:3300/0 conn(0x7f46f406ccc0 msgr2=0x7f46f40b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:25.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.694+0000 7f46ebfff700 1 --2- 172.21.15.43:0/498737200 >> v2:172.21.15.178:3300/0 conn(0x7f46f406ccc0 0x7f46f40b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.694+0000 7f46ebfff700 1 -- 172.21.15.43:0/498737200 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f41604f0 con 0x7f46f40af060 2023-06-28T20:08:25.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.694+0000 7f46ebfff700 1 -- 172.21.15.43:0/498737200 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f46f0001670 con 0x7f46f40af060 2023-06-28T20:08:25.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.694+0000 7f46ebfff700 1 -- 172.21.15.43:0/498737200 >> v1:172.21.15.43:6789/0 conn(0x7f46f40af060 legacy=0x7f46f4070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:25.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.694+0000 7f46ebfff700 1 --2- 172.21.15.43:0/498737200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46e0003a00 0x7f46e0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:25.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.694+0000 7f46ebfff700 1 --2- 172.21.15.43:0/498737200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46e0004430 0x7f46e00048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:25.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.694+0000 7f46ebfff700 1 --2- 172.21.15.43:0/498737200 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46f40aec90 0x7f46e0004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:25.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.694+0000 7f46ebfff700 1 -- 172.21.15.43:0/498737200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f415cb30 con 0x7f46e0004430 2023-06-28T20:08:25.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.694+0000 7f46ebfff700 1 -- 172.21.15.43:0/498737200 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f4071410 con 0x7f46e0003a00 2023-06-28T20:08:25.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.694+0000 7f46ebfff700 1 -- 172.21.15.43:0/498737200 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e000ba10 con 0x7f46f40aec90 2023-06-28T20:08:25.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.694+0000 7f46f8a25700 1 --2- 172.21.15.43:0/498737200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46e0004430 0x7f46e00048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:25.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.694+0000 7f46f9226700 1 --2- 172.21.15.43:0/498737200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46e0003a00 0x7f46e0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:25.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.694+0000 7f46f9a27700 1 --2- 172.21.15.43:0/498737200 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46f40aec90 0x7f46e0004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:25.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.695+0000 7f46f8a25700 1 -- 172.21.15.43:0/498737200 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46f40aec90 msgr2=0x7f46e0004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:25.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.695+0000 7f46f8a25700 1 --2- 172.21.15.43:0/498737200 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46f40aec90 0x7f46e0004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.695+0000 7f46f8a25700 1 -- 172.21.15.43:0/498737200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46e0003a00 msgr2=0x7f46e0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:25.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.695+0000 7f46f8a25700 1 --2- 172.21.15.43:0/498737200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46e0003a00 0x7f46e0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.695+0000 7f46f8a25700 1 -- 172.21.15.43:0/498737200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e00035b0 con 0x7f46e0004430 2023-06-28T20:08:25.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.695+0000 7f46f8a25700 1 --2- 172.21.15.43:0/498737200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46e0004430 0x7f46e00048f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f46f000c2f0 tx=0x7f46e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642943de11a9a59 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:25.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.695+0000 7f46ebfff700 1 -- 172.21.15.43:0/498737200 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0007c40 con 0x7f46e0004430 2023-06-28T20:08:25.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.695+0000 7f46ebfff700 1 -- 172.21.15.43:0/498737200 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f000ce30 con 0x7f46e0004430 2023-06-28T20:08:25.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.695+0000 7f46ebfff700 1 -- 172.21.15.43:0/498737200 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0005ba0 con 0x7f46e0004430 2023-06-28T20:08:25.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.695+0000 7f46fb48a700 1 -- 172.21.15.43:0/498737200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46e0004430 msgr2=0x7f46e00048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:25.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.695+0000 7f46fb48a700 1 --2- 172.21.15.43:0/498737200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46e0004430 0x7f46e00048f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f46f000c2f0 tx=0x7f46e0005880 comp rx=0 tx=0).stop 2023-06-28T20:08:25.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.695+0000 7f46f9a27700 1 -- 172.21.15.43:0/498737200 reap_dead start 2023-06-28T20:08:25.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.695+0000 7f46fb48a700 1 -- 172.21.15.43:0/498737200 shutdown_connections 2023-06-28T20:08:25.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.695+0000 7f46fb48a700 1 -- 172.21.15.43:0/498737200 >> 172.21.15.43:0/498737200 conn(0x7f46f406b000 msgr2=0x7f46f406b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:25.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.696+0000 7f46fb48a700 1 -- 172.21.15.43:0/498737200 shutdown_connections 2023-06-28T20:08:25.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.696+0000 7f46fb48a700 1 -- 172.21.15.43:0/498737200 wait complete. 2023-06-28T20:08:25.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.697+0000 7f46fb48a700 1 Processor -- start 2023-06-28T20:08:25.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.697+0000 7f46fb48a700 1 -- start start 2023-06-28T20:08:25.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.697+0000 7f46fb48a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46e0004430 0x7f46f40c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:25.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.697+0000 7f46fb48a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46f40b05e0 0x7f46f40b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:25.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.697+0000 7f46fb48a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46f41f1640 0x7f46f41f1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:25.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.697+0000 7f46fb48a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f40c7100 con 0x7f46e0004430 2023-06-28T20:08:25.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.697+0000 7f46fb48a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f40b0f90 con 0x7f46f40b05e0 2023-06-28T20:08:25.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.697+0000 7f46fb48a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46f40c6970 con 0x7f46f41f1640 2023-06-28T20:08:25.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.698+0000 7f46f9226700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46e0004430 0x7f46f40c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:25.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.698+0000 7f46f9226700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46e0004430 0x7f46f40c6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55080/0 (socket says 172.21.15.43:55080) 2023-06-28T20:08:25.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.698+0000 7f46f9226700 1 -- 172.21.15.43:0/3100484036 learned_addr learned my addr 172.21.15.43:0/3100484036 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:25.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.698+0000 7f46f8a25700 1 --2- 172.21.15.43:0/3100484036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46f40b05e0 0x7f46f40b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:25.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.698+0000 7f46f9a27700 1 --2- 172.21.15.43:0/3100484036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46f41f1640 0x7f46f41f1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:25.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.698+0000 7f46f9226700 1 -- 172.21.15.43:0/3100484036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46f41f1640 msgr2=0x7f46f41f1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:25.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.698+0000 7f46f9226700 1 --2- 172.21.15.43:0/3100484036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46f41f1640 0x7f46f41f1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.698+0000 7f46f9226700 1 -- 172.21.15.43:0/3100484036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46f40b05e0 msgr2=0x7f46f40b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:25.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.698+0000 7f46f9226700 1 --2- 172.21.15.43:0/3100484036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46f40b05e0 0x7f46f40b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.698+0000 7f46f9226700 1 -- 172.21.15.43:0/3100484036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f41f3130 con 0x7f46e0004430 2023-06-28T20:08:25.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.698+0000 7f46f9226700 1 --2- 172.21.15.43:0/3100484036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46e0004430 0x7f46f40c6bc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f46e40119d0 tx=0x7f46e4011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:25.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.698+0000 7f46e9ffb700 1 -- 172.21.15.43:0/3100484036 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e4011e70 con 0x7f46e0004430 2023-06-28T20:08:25.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.698+0000 7f46e9ffb700 1 -- 172.21.15.43:0/3100484036 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e400f8e0 con 0x7f46e0004430 2023-06-28T20:08:25.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.698+0000 7f46fb48a700 1 -- 172.21.15.43:0/3100484036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f41f2550 con 0x7f46e0004430 2023-06-28T20:08:25.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.698+0000 7f46e9ffb700 1 -- 172.21.15.43:0/3100484036 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e400fc70 con 0x7f46e0004430 2023-06-28T20:08:25.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.699+0000 7f46fb48a700 1 -- 172.21.15.43:0/3100484036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f41f2b10 con 0x7f46e0004430 2023-06-28T20:08:25.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.699+0000 7f46fb48a700 1 -- 172.21.15.43:0/3100484036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f40728b0 con 0x7f46e0004430 2023-06-28T20:08:25.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.700+0000 7f46e9ffb700 1 -- 172.21.15.43:0/3100484036 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f46e4014440 con 0x7f46e0004430 2023-06-28T20:08:25.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.700+0000 7f46e9ffb700 1 --2- 172.21.15.43:0/3100484036 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f46d806ace0 0x7f46d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:25.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.701+0000 7f46e9ffb700 1 -- 172.21.15.43:0/3100484036 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f46e4089860 con 0x7f46e0004430 2023-06-28T20:08:25.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.701+0000 7f46f8a25700 1 --2- 172.21.15.43:0/3100484036 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f46d806ace0 0x7f46d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:25.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.701+0000 7f46f8a25700 1 --2- 172.21.15.43:0/3100484036 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f46d806ace0 0x7f46d806d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f46f0001930 tx=0x7f46f000a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:25.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.705+0000 7f46e9ffb700 1 -- 172.21.15.43:0/3100484036 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f46e40549a0 con 0x7f46e0004430 2023-06-28T20:08:25.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.985+0000 7f46fb48a700 1 -- 172.21.15.43:0/3100484036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f46f406d9c0 con 0x7f46e0004430 2023-06-28T20:08:25.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.989+0000 7f46e9ffb700 1 -- 172.21.15.43:0/3100484036 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f46e4057fc0 con 0x7f46e0004430 2023-06-28T20:08:25.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.991+0000 7f46fb48a700 1 -- 172.21.15.43:0/3100484036 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f46d806ace0 msgr2=0x7f46d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:25.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.991+0000 7f46fb48a700 1 --2- 172.21.15.43:0/3100484036 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f46d806ace0 0x7f46d806d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f46f0001930 tx=0x7f46f000a000 comp rx=0 tx=0).stop 2023-06-28T20:08:25.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.991+0000 7f46fb48a700 1 -- 172.21.15.43:0/3100484036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46e0004430 msgr2=0x7f46f40c6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:25.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.991+0000 7f46fb48a700 1 --2- 172.21.15.43:0/3100484036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46e0004430 0x7f46f40c6bc0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f46e40119d0 tx=0x7f46e4011ce0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.991+0000 7f46fb48a700 1 -- 172.21.15.43:0/3100484036 shutdown_connections 2023-06-28T20:08:25.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.991+0000 7f46fb48a700 1 --2- 172.21.15.43:0/3100484036 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f46d806ace0 0x7f46d806d190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.991+0000 7f46fb48a700 1 --2- 172.21.15.43:0/3100484036 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f46f41f1640 0x7f46f41f1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.991+0000 7f46fb48a700 1 --2- 172.21.15.43:0/3100484036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46e0004430 0x7f46f40c6bc0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.991+0000 7f46fb48a700 1 --2- 172.21.15.43:0/3100484036 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f46f40b05e0 0x7f46f40b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:25.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.991+0000 7f46fb48a700 1 -- 172.21.15.43:0/3100484036 >> 172.21.15.43:0/3100484036 conn(0x7f46f406b000 msgr2=0x7f46f40ce590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:25.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.991+0000 7f46fb48a700 1 -- 172.21.15.43:0/3100484036 shutdown_connections 2023-06-28T20:08:25.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:25.991+0000 7f46fb48a700 1 -- 172.21.15.43:0/3100484036 wait complete. 2023-06-28T20:08:25.998 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2023-06-28T20:08:26.007 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-28T20:08:26.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.128+0000 7fbadec13700 1 Processor -- start 2023-06-28T20:08:26.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.128+0000 7fbadec13700 1 -- start start 2023-06-28T20:08:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.129+0000 7fbadec13700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbad80d10f0 0x7fbad80ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.129+0000 7fbadec13700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbad80cedf0 0x7fbad80cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.129+0000 7fbadec13700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad815ab50 con 0x7fbad80cb8e0 2023-06-28T20:08:26.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.129+0000 7fbadec13700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad80cf260 con 0x7fbad80cedf0 2023-06-28T20:08:26.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.129+0000 7fbadec13700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad80cd1b0 con 0x7fbad80d10f0 2023-06-28T20:08:26.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.129+0000 7fbad7fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbad80cedf0 0x7fbad80cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:26.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.129+0000 7fbadd1b0700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbad80cb8e0 0x7fbad80cd600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44278/0 (socket says 172.21.15.43:44278) 2023-06-28T20:08:26.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.129+0000 7fbadd1b0700 1 -- 172.21.15.43:0/202603020 learned_addr learned my addr 172.21.15.43:0/202603020 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:26.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.129+0000 7fbadc9af700 1 --2- 172.21.15.43:0/202603020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbad80d10f0 0x7fbad80ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:26.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.129+0000 7fbad7fff700 1 -- 172.21.15.43:0/202603020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbad80d10f0 msgr2=0x7fbad80ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:26.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.129+0000 7fbad7fff700 1 --2- 172.21.15.43:0/202603020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbad80d10f0 0x7fbad80ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.129+0000 7fbad7fff700 1 -- 172.21.15.43:0/202603020 >> v1:172.21.15.43:6789/0 conn(0x7fbad80cb8e0 legacy=0x7fbad80cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:26.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.129+0000 7fbad7fff700 1 -- 172.21.15.43:0/202603020 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad80cc110 con 0x7fbad80cedf0 2023-06-28T20:08:26.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.129+0000 7fbad77fe700 1 -- 172.21.15.43:0/202603020 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 988585833 0 0) 0x7fbad815ab50 con 0x7fbad80cb8e0 2023-06-28T20:08:26.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.129+0000 7fbadc9af700 1 --2- 172.21.15.43:0/202603020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbad80d10f0 0x7fbad80ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:26.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.130+0000 7fbad7fff700 1 --2- 172.21.15.43:0/202603020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbad80cedf0 0x7fbad80cb3a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009840 tx=0x7fbac4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43348e96d2298689 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:26.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.130+0000 7fbad77fe700 1 -- 172.21.15.43:0/202603020 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac4011830 con 0x7fbad80cedf0 2023-06-28T20:08:26.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.130+0000 7fbad77fe700 1 -- 172.21.15.43:0/202603020 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac4011990 con 0x7fbad80cedf0 2023-06-28T20:08:26.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.130+0000 7fbad77fe700 1 -- 172.21.15.43:0/202603020 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac4011d20 con 0x7fbad80cedf0 2023-06-28T20:08:26.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.130+0000 7fbadec13700 1 -- 172.21.15.43:0/202603020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbad80cedf0 msgr2=0x7fbad80cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:26.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.130+0000 7fbadec13700 1 --2- 172.21.15.43:0/202603020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbad80cedf0 0x7fbad80cb3a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009840 tx=0x7fbac4009b50 comp rx=0 tx=0).stop 2023-06-28T20:08:26.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.130+0000 7fbadec13700 1 -- 172.21.15.43:0/202603020 shutdown_connections 2023-06-28T20:08:26.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.130+0000 7fbadec13700 1 --2- 172.21.15.43:0/202603020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbad80d10f0 0x7fbad80ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.130+0000 7fbadec13700 1 --2- 172.21.15.43:0/202603020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbad80cedf0 0x7fbad80cb3a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.130+0000 7fbadec13700 1 -- 172.21.15.43:0/202603020 >> 172.21.15.43:0/202603020 conn(0x7fbad80664b0 msgr2=0x7fbad809a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:26.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.130+0000 7fbadec13700 1 -- 172.21.15.43:0/202603020 shutdown_connections 2023-06-28T20:08:26.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.130+0000 7fbadec13700 1 -- 172.21.15.43:0/202603020 wait complete. 2023-06-28T20:08:26.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.132+0000 7fbadec13700 1 Processor -- start 2023-06-28T20:08:26.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.132+0000 7fbadec13700 1 -- start start 2023-06-28T20:08:26.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.132+0000 7fbadec13700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbad80cb8e0 0x7fbad81e4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.140 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.132+0000 7fbadec13700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbad80cedf0 0x7fbad815c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.132+0000 7fbadec13700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbad80d10f0 0x7fbad80b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.132+0000 7fbadec13700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad815d840 con 0x7fbad80d10f0 2023-06-28T20:08:26.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.132+0000 7fbadec13700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad815cc30 con 0x7fbad80cedf0 2023-06-28T20:08:26.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.132+0000 7fbadec13700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad815cf10 con 0x7fbad80cb8e0 2023-06-28T20:08:26.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.133+0000 7fbadd1b0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbad80d10f0 0x7fbad80b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:26.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.133+0000 7fbadd1b0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbad80d10f0 0x7fbad80b4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55098/0 (socket says 172.21.15.43:55098) 2023-06-28T20:08:26.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.133+0000 7fbadd1b0700 1 -- 172.21.15.43:0/3588777675 learned_addr learned my addr 172.21.15.43:0/3588777675 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:26.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.133+0000 7fbadc9af700 1 --2- 172.21.15.43:0/3588777675 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbad80cb8e0 0x7fbad81e4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:26.142 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.133+0000 7fbad7fff700 1 --2- 172.21.15.43:0/3588777675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbad80cedf0 0x7fbad815c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:26.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.133+0000 7fbadd1b0700 1 -- 172.21.15.43:0/3588777675 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbad80cb8e0 msgr2=0x7fbad81e4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:26.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.133+0000 7fbadd1b0700 1 --2- 172.21.15.43:0/3588777675 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbad80cb8e0 0x7fbad81e4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.133+0000 7fbadd1b0700 1 -- 172.21.15.43:0/3588777675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbad80cedf0 msgr2=0x7fbad815c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:26.143 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.133+0000 7fbadd1b0700 1 --2- 172.21.15.43:0/3588777675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbad80cedf0 0x7fbad815c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.133+0000 7fbadd1b0700 1 -- 172.21.15.43:0/3588777675 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac40094a0 con 0x7fbad80d10f0 2023-06-28T20:08:26.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.133+0000 7fbad7fff700 1 --2- 172.21.15.43:0/3588777675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbad80cedf0 0x7fbad815c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:26.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.133+0000 7fbadc9af700 1 --2- 172.21.15.43:0/3588777675 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbad80cb8e0 0x7fbad81e4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:26.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.133+0000 7fbadd1b0700 1 --2- 172.21.15.43:0/3588777675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbad80d10f0 0x7fbad80b4a00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbad000cca0 tx=0x7fbad000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:26.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.133+0000 7fbad57fa700 1 -- 172.21.15.43:0/3588777675 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad000cea0 con 0x7fbad80d10f0 2023-06-28T20:08:26.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.133+0000 7fbad57fa700 1 -- 172.21.15.43:0/3588777675 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad000ade0 con 0x7fbad80d10f0 2023-06-28T20:08:26.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.133+0000 7fbadec13700 1 -- 172.21.15.43:0/3588777675 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad815d1f0 con 0x7fbad80d10f0 2023-06-28T20:08:26.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.133+0000 7fbad57fa700 1 -- 172.21.15.43:0/3588777675 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad0007310 con 0x7fbad80d10f0 2023-06-28T20:08:26.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.133+0000 7fbadec13700 1 -- 172.21.15.43:0/3588777675 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad81e4a80 con 0x7fbad80d10f0 2023-06-28T20:08:26.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.134+0000 7fbadec13700 1 -- 172.21.15.43:0/3588777675 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad815d9b0 con 0x7fbad80d10f0 2023-06-28T20:08:26.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.135+0000 7fbad57fa700 1 -- 172.21.15.43:0/3588777675 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbad0007470 con 0x7fbad80d10f0 2023-06-28T20:08:26.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.135+0000 7fbad57fa700 1 --2- 172.21.15.43:0/3588777675 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbabc06ace0 0x7fbabc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.135+0000 7fbadc9af700 1 --2- 172.21.15.43:0/3588777675 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbabc06ace0 0x7fbabc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:26.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.135+0000 7fbad57fa700 1 -- 172.21.15.43:0/3588777675 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbad00893d0 con 0x7fbad80d10f0 2023-06-28T20:08:26.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.136+0000 7fbadc9af700 1 --2- 172.21.15.43:0/3588777675 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbabc06ace0 0x7fbabc06d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00aa10 tx=0x7fbacc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:26.148 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.140+0000 7fbad57fa700 1 -- 172.21.15.43:0/3588777675 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbad0054480 con 0x7fbad80d10f0 2023-06-28T20:08:26.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.385+0000 7fbadec13700 1 -- 172.21.15.43:0/3588777675 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbad80728b0 con 0x7fbad80d10f0 2023-06-28T20:08:26.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.385+0000 7fbad57fa700 1 -- 172.21.15.43:0/3588777675 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4674 (secure 0 0 0) 0x7fbad0057aa0 con 0x7fbad80d10f0 2023-06-28T20:08:26.390 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:26.390 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1274138939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":1274138939},{"type":"v1","addr":"172.21.15.178:6835","nonce":1274138939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3882460026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3882460026},{"type":"v1","addr":"172.21.15.178:6834","nonce":3882460026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/382468709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":382468709},{"type":"v1","addr":"172.21.15.43:6836","nonce":382468709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-28T20:08:20.073546+0000","modified":"2023-06-28T20:08:23.914623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5112},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5112":{"gid":5112,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6837/1096784002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":1096784002},{"type":"v1","addr":"172.21.15.43:6837","nonce":1096784002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-06-28T20:08:26.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.387+0000 7fbadec13700 1 -- 172.21.15.43:0/3588777675 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbabc06ace0 msgr2=0x7fbabc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:26.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.387+0000 7fbadec13700 1 --2- 172.21.15.43:0/3588777675 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbabc06ace0 0x7fbabc06d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbacc00aa10 tx=0x7fbacc008040 comp rx=0 tx=0).stop 2023-06-28T20:08:26.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.387+0000 7fbadec13700 1 -- 172.21.15.43:0/3588777675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbad80d10f0 msgr2=0x7fbad80b4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:26.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.387+0000 7fbadec13700 1 --2- 172.21.15.43:0/3588777675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbad80d10f0 0x7fbad80b4a00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbad000cca0 tx=0x7fbad000a340 comp rx=0 tx=0).stop 2023-06-28T20:08:26.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.387+0000 7fbadec13700 1 -- 172.21.15.43:0/3588777675 shutdown_connections 2023-06-28T20:08:26.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.387+0000 7fbadec13700 1 --2- 172.21.15.43:0/3588777675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbad80d10f0 0x7fbad80b4a00 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.387+0000 7fbadec13700 1 --2- 172.21.15.43:0/3588777675 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbabc06ace0 0x7fbabc06d190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.387+0000 7fbadec13700 1 --2- 172.21.15.43:0/3588777675 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbad80cb8e0 0x7fbad81e4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.387+0000 7fbadec13700 1 --2- 172.21.15.43:0/3588777675 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbad80cedf0 0x7fbad815c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.387+0000 7fbadec13700 1 -- 172.21.15.43:0/3588777675 >> 172.21.15.43:0/3588777675 conn(0x7fbad80664b0 msgr2=0x7fbad80ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:26.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.387+0000 7fbadec13700 1 -- 172.21.15.43:0/3588777675 shutdown_connections 2023-06-28T20:08:26.393 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.387+0000 7fbadec13700 1 -- 172.21.15.43:0/3588777675 wait complete. 2023-06-28T20:08:26.393 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 15 2023-06-28T20:08:26.806 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-06-28T20:08:20.073546+0000', 'modified': '2023-06-28T20:08:23.914623+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5112}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5112': {'gid': 5112, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.43:6837/1096784002', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6835', 'nonce': 1096784002}, {'type': 'v1', 'addr': '172.21.15.43:6837', 'nonce': 1096784002}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-06-28T20:08:26.807 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-06-28T20:08:26.807 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-06-28T20:08:26.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.935+0000 7f819082f700 1 Processor -- start 2023-06-28T20:08:26.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.935+0000 7f819082f700 1 -- start start 2023-06-28T20:08:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.936+0000 7f819082f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f818c0af060 0x7f818c160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.936+0000 7f819082f700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f818c06ccc0 0x7f818c0b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.936+0000 7f819082f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818c160250 con 0x7f818c0aec90 2023-06-28T20:08:26.944 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.936+0000 7f819082f700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f818c15cb30 con 0x7f818c0af060 2023-06-28T20:08:26.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.936+0000 7f819082f700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f818c071410 con 0x7f818c06ccc0 2023-06-28T20:08:26.945 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.936+0000 7f8189d9b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f818c0af060 0x7f818c160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:26.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.936+0000 7f818a59c700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f818c0aec90 0x7f818c070bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44288/0 (socket says 172.21.15.43:44288) 2023-06-28T20:08:26.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.936+0000 7f818a59c700 1 -- 172.21.15.43:0/2047807317 learned_addr learned my addr 172.21.15.43:0/2047807317 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:26.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.937+0000 7f818ad9d700 1 --2- 172.21.15.43:0/2047807317 >> v2:172.21.15.178:3300/0 conn(0x7f818c06ccc0 0x7f818c0b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:26.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.937+0000 7f8189d9b700 1 -- 172.21.15.43:0/2047807317 >> v2:172.21.15.178:3300/0 conn(0x7f818c06ccc0 msgr2=0x7f818c0b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:26.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.937+0000 7f8189d9b700 1 --2- 172.21.15.43:0/2047807317 >> v2:172.21.15.178:3300/0 conn(0x7f818c06ccc0 0x7f818c0b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.937+0000 7f8189d9b700 1 -- 172.21.15.43:0/2047807317 >> v1:172.21.15.43:6789/0 conn(0x7f818c0aec90 legacy=0x7f818c070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:26.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.937+0000 7f8189d9b700 1 -- 172.21.15.43:0/2047807317 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818c1604f0 con 0x7f818c0af060 2023-06-28T20:08:26.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.937+0000 7f818ad9d700 1 --2- 172.21.15.43:0/2047807317 >> v2:172.21.15.178:3300/0 conn(0x7f818c06ccc0 0x7f818c0b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:26.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.937+0000 7f818959a700 1 -- 172.21.15.43:0/2047807317 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112289518 0 0) 0x7f818c160250 con 0x7f818c0aec90 2023-06-28T20:08:26.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.937+0000 7f8189d9b700 1 --2- 172.21.15.43:0/2047807317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f818c0af060 0x7f818c160cf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f818c0cd8a0 tx=0x7f817c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=627a644f98b75abd server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:26.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.937+0000 7f818959a700 1 -- 172.21.15.43:0/2047807317 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c011830 con 0x7f818c0af060 2023-06-28T20:08:26.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.937+0000 7f818959a700 1 -- 172.21.15.43:0/2047807317 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817c011990 con 0x7f818c0af060 2023-06-28T20:08:26.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.937+0000 7f818959a700 1 -- 172.21.15.43:0/2047807317 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c011d20 con 0x7f818c0af060 2023-06-28T20:08:26.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.938+0000 7f819082f700 1 -- 172.21.15.43:0/2047807317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f818c0af060 msgr2=0x7f818c160cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:26.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.938+0000 7f819082f700 1 --2- 172.21.15.43:0/2047807317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f818c0af060 0x7f818c160cf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f818c0cd8a0 tx=0x7f817c009b20 comp rx=0 tx=0).stop 2023-06-28T20:08:26.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.938+0000 7f819082f700 1 -- 172.21.15.43:0/2047807317 shutdown_connections 2023-06-28T20:08:26.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.938+0000 7f819082f700 1 --2- 172.21.15.43:0/2047807317 >> v2:172.21.15.178:3300/0 conn(0x7f818c06ccc0 0x7f818c0b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.938+0000 7f819082f700 1 --2- 172.21.15.43:0/2047807317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f818c0af060 0x7f818c160cf0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.938+0000 7f819082f700 1 -- 172.21.15.43:0/2047807317 >> 172.21.15.43:0/2047807317 conn(0x7f818c06b000 msgr2=0x7f818c06b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:26.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.938+0000 7f819082f700 1 -- 172.21.15.43:0/2047807317 shutdown_connections 2023-06-28T20:08:26.951 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.938+0000 7f819082f700 1 -- 172.21.15.43:0/2047807317 wait complete. 2023-06-28T20:08:26.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.939+0000 7f819082f700 1 Processor -- start 2023-06-28T20:08:26.952 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.939+0000 7f819082f700 1 -- start start 2023-06-28T20:08:26.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.940+0000 7f819082f700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f818c06ccc0 0x7f818c1560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.940+0000 7f819082f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f818c0aec90 0x7f818c15d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.940+0000 7f819082f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f818c0af060 0x7f818c1558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.940+0000 7f819082f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c15dac0 con 0x7f818c0af060 2023-06-28T20:08:26.959 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.940+0000 7f819082f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818c15dd20 con 0x7f818c0aec90 2023-06-28T20:08:26.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.940+0000 7f819082f700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818c15e000 con 0x7f818c06ccc0 2023-06-28T20:08:26.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.940+0000 7f818ad9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f818c0af060 0x7f818c1558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:26.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.940+0000 7f818ad9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f818c0af060 0x7f818c1558c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55112/0 (socket says 172.21.15.43:55112) 2023-06-28T20:08:26.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.940+0000 7f818ad9d700 1 -- 172.21.15.43:0/94319426 learned_addr learned my addr 172.21.15.43:0/94319426 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:26.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.940+0000 7f8189d9b700 1 --2- 172.21.15.43:0/94319426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f818c0aec90 0x7f818c15d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:26.960 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.940+0000 7f818a59c700 1 --2- 172.21.15.43:0/94319426 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f818c06ccc0 0x7f818c1560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:26.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.940+0000 7f818ad9d700 1 -- 172.21.15.43:0/94319426 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f818c06ccc0 msgr2=0x7f818c1560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:26.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.940+0000 7f818ad9d700 1 --2- 172.21.15.43:0/94319426 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f818c06ccc0 0x7f818c1560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.940+0000 7f818ad9d700 1 -- 172.21.15.43:0/94319426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f818c0aec90 msgr2=0x7f818c15d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:26.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.940+0000 7f818ad9d700 1 --2- 172.21.15.43:0/94319426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f818c0aec90 0x7f818c15d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.940+0000 7f818ad9d700 1 -- 172.21.15.43:0/94319426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817c0094a0 con 0x7f818c0af060 2023-06-28T20:08:26.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.941+0000 7f818ad9d700 1 --2- 172.21.15.43:0/94319426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f818c0af060 0x7f818c1558c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f818400cef0 tx=0x7f818400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:26.961 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.941+0000 7f817affd700 1 -- 172.21.15.43:0/94319426 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818400ad00 con 0x7f818c0af060 2023-06-28T20:08:26.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.941+0000 7f817affd700 1 -- 172.21.15.43:0/94319426 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818400ae60 con 0x7f818c0af060 2023-06-28T20:08:26.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.941+0000 7f819082f700 1 -- 172.21.15.43:0/94319426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818c159110 con 0x7f818c0af060 2023-06-28T20:08:26.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.941+0000 7f817affd700 1 -- 172.21.15.43:0/94319426 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8184015680 con 0x7f818c0af060 2023-06-28T20:08:26.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.941+0000 7f819082f700 1 -- 172.21.15.43:0/94319426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818c155e00 con 0x7f818c0af060 2023-06-28T20:08:26.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.944+0000 7f817affd700 1 -- 172.21.15.43:0/94319426 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f818400d430 con 0x7f818c0af060 2023-06-28T20:08:26.962 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.944+0000 7f817affd700 1 --2- 172.21.15.43:0/94319426 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f816c06ace0 0x7f816c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.944+0000 7f817affd700 1 -- 172.21.15.43:0/94319426 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f81840898a0 con 0x7f818c0af060 2023-06-28T20:08:26.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.944+0000 7f818a59c700 1 --2- 172.21.15.43:0/94319426 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f816c06ace0 0x7f816c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:26.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.945+0000 7f818a59c700 1 --2- 172.21.15.43:0/94319426 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f816c06ace0 0x7f816c06d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8180007fd0 tx=0x7f8180007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:26.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.945+0000 7f819c9b3b80 1 Processor -- start 2023-06-28T20:08:26.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.945+0000 7f819c9b3b80 1 -- start start 2023-06-28T20:08:26.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.946+0000 7f819c9b3b80 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fef75a2170 0x55fef75c9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.963 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.946+0000 7f819c9b3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fef75a2540 0x55fef75acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.946+0000 7f819c9b3b80 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef757d6c0 0x55fef75ad470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.946+0000 7f819c9b3b80 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fef7487c60 con 0x55fef757d6c0 2023-06-28T20:08:26.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.946+0000 7f819c9b3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fef74da9f0 con 0x55fef75a2540 2023-06-28T20:08:26.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.946+0000 7f819c9b3b80 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fef74d3e80 con 0x55fef75a2170 2023-06-28T20:08:26.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.946+0000 7f818ad9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef757d6c0 0x55fef75ad470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:26.964 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.946+0000 7f818ad9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef757d6c0 0x55fef75ad470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55120/0 (socket says 172.21.15.43:55120) 2023-06-28T20:08:26.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.946+0000 7f818ad9d700 1 -- 172.21.15.43:0/3070419516 learned_addr learned my addr 172.21.15.43:0/3070419516 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:26.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.946+0000 7f818a59c700 1 --2- 172.21.15.43:0/3070419516 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fef75a2170 0x55fef75c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:26.965 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.946+0000 7f8189d9b700 1 --2- 172.21.15.43:0/3070419516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fef75a2540 0x55fef75acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:26.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.946+0000 7f818ad9d700 1 -- 172.21.15.43:0/3070419516 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fef75a2170 msgr2=0x55fef75c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:26.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.946+0000 7f818ad9d700 1 --2- 172.21.15.43:0/3070419516 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fef75a2170 0x55fef75c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.946+0000 7f818ad9d700 1 -- 172.21.15.43:0/3070419516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fef75a2540 msgr2=0x55fef75acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:26.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.946+0000 7f818ad9d700 1 --2- 172.21.15.43:0/3070419516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fef75a2540 0x55fef75acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.946+0000 7f818ad9d700 1 -- 172.21.15.43:0/3070419516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fef7301d90 con 0x55fef757d6c0 2023-06-28T20:08:26.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.946+0000 7f818ad9d700 1 --2- 172.21.15.43:0/3070419516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef757d6c0 0x55fef75ad470 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f81840604c0 tx=0x7f818405c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=408323c4c1372d12 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:26.966 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.946+0000 7f816bfff700 1 -- 172.21.15.43:0/3070419516 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8184056a40 con 0x55fef757d6c0 2023-06-28T20:08:26.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.946+0000 7f816bfff700 1 -- 172.21.15.43:0/3070419516 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8184060560 con 0x55fef757d6c0 2023-06-28T20:08:26.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.946+0000 7f816bfff700 1 -- 172.21.15.43:0/3070419516 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818405c170 con 0x55fef757d6c0 2023-06-28T20:08:26.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.947+0000 7f819c9b3b80 1 -- 172.21.15.43:0/3070419516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef757d6c0 msgr2=0x55fef75ad470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:26.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.947+0000 7f819c9b3b80 1 --2- 172.21.15.43:0/3070419516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef757d6c0 0x55fef75ad470 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f81840604c0 tx=0x7f818405c730 comp rx=0 tx=0).stop 2023-06-28T20:08:26.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.947+0000 7f819c9b3b80 1 -- 172.21.15.43:0/3070419516 shutdown_connections 2023-06-28T20:08:26.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.947+0000 7f819c9b3b80 1 --2- 172.21.15.43:0/3070419516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef757d6c0 0x55fef75ad470 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.967 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.947+0000 7f819c9b3b80 1 --2- 172.21.15.43:0/3070419516 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fef75a2170 0x55fef75c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.947+0000 7f819c9b3b80 1 --2- 172.21.15.43:0/3070419516 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fef75a2540 0x55fef75acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.947+0000 7f819c9b3b80 1 -- 172.21.15.43:0/3070419516 >> 172.21.15.43:0/3070419516 conn(0x55fef7577a30 msgr2=0x55fef757cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:26.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.947+0000 7f819c9b3b80 1 -- 172.21.15.43:0/3070419516 shutdown_connections 2023-06-28T20:08:26.968 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.947+0000 7f819c9b3b80 1 -- 172.21.15.43:0/3070419516 wait complete. 2023-06-28T20:08:26.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.948+0000 7f819c9b3b80 1 -- start start 2023-06-28T20:08:26.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.948+0000 7f819c9b3b80 1 Processor -- start 2023-06-28T20:08:26.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.949+0000 7f819c9b3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fef757d6c0 0x55fef75a2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.969 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.949+0000 7f819c9b3b80 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef75a2170 0x55fef75c0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.949+0000 7f819c9b3b80 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fef75a2540 0x55fef75c05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.949+0000 7f819c9b3b80 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fef7473a70 con 0x55fef75a2170 2023-06-28T20:08:26.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.949+0000 7f819c9b3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fef7473770 con 0x55fef757d6c0 2023-06-28T20:08:26.970 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.949+0000 7f819c9b3b80 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fef73578d0 con 0x55fef75a2540 2023-06-28T20:08:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.949+0000 7f819c9b3b80 10 client.0 _ll_register_callbacks cb 0x55fef7471710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-28T20:08:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.949+0000 7f8189d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef75a2170 0x55fef75c0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.949+0000 7f8189d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef75a2170 0x55fef75c0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55124/0 (socket says 172.21.15.43:55124) 2023-06-28T20:08:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.949+0000 7f8189d9b700 1 -- 172.21.15.43:0/2373065556 learned_addr learned my addr 172.21.15.43:0/2373065556 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:26.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.949+0000 7f818a59c700 1 --2- 172.21.15.43:0/2373065556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fef757d6c0 0x55fef75a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:26.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.949+0000 7f8189d9b700 1 -- 172.21.15.43:0/2373065556 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fef75a2540 msgr2=0x55fef75c05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:26.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.949+0000 7f818ad9d700 1 --2- 172.21.15.43:0/2373065556 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fef75a2540 0x55fef75c05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:26.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.949+0000 7f8189d9b700 1 --2- 172.21.15.43:0/2373065556 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fef75a2540 0x55fef75c05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.949+0000 7f8189d9b700 1 -- 172.21.15.43:0/2373065556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fef757d6c0 msgr2=0x55fef75a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:26.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.949+0000 7f8189d9b700 1 --2- 172.21.15.43:0/2373065556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fef757d6c0 0x55fef75a2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.972 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.949+0000 7f8189d9b700 1 -- 172.21.15.43:0/2373065556 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818c155e00 con 0x55fef75a2170 2023-06-28T20:08:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.949+0000 7f818a59c700 1 --2- 172.21.15.43:0/2373065556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fef757d6c0 0x55fef75a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.949+0000 7f818ad9d700 1 --2- 172.21.15.43:0/2373065556 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fef75a2540 0x55fef75c05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.949+0000 7f8189d9b700 1 --2- 172.21.15.43:0/2373065556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef75a2170 0x55fef75c0060 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f817c013000 tx=0x7f817c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1de49f04a1a86b9 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.949+0000 7f81697fa700 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-06-28T20:08:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.950+0000 7f81697fa700 1 -- 172.21.15.43:0/2373065556 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c004450 con 0x55fef75a2170 2023-06-28T20:08:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.950+0000 7f81697fa700 1 -- 172.21.15.43:0/2373065556 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f817c01f040 con 0x55fef75a2170 2023-06-28T20:08:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.950+0000 7f819c9b3b80 1 -- 172.21.15.43:0/2373065556 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fef7575040 con 0x55fef75a2170 2023-06-28T20:08:26.973 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.950+0000 7f81697fa700 1 -- 172.21.15.43:0/2373065556 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f817c0045b0 con 0x55fef75a2170 2023-06-28T20:08:26.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.950+0000 7f819c9b3b80 10 client.5277 fetch_fsmap learned FSMap version 15 2023-06-28T20:08:26.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.950+0000 7f819c9b3b80 1 -- 172.21.15.43:0/2373065556 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x55fef72eac10 con 0x55fef75a2170 2023-06-28T20:08:26.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.950+0000 7f81697fa700 1 -- 172.21.15.43:0/2373065556 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fef7575040 con 0x55fef75a2170 2023-06-28T20:08:26.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.950+0000 7f81697fa700 1 -- 172.21.15.43:0/2373065556 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 15) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f817c02e430 con 0x55fef75a2170 2023-06-28T20:08:26.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.950+0000 7f819c9b3b80 10 client.5277 fetch_fsmap finished waiting for FSMap version 15 2023-06-28T20:08:26.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.950+0000 7f819c9b3b80 10 client.5277 resolve_mds: resolved name 'a' to daemon mds.a 2023-06-28T20:08:26.974 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.950+0000 7f819c9b3b80 20 client.5277 populate_metadata read hostname 'smithi043' 2023-06-28T20:08:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.950+0000 7f819c9b3b80 1 --2- 172.21.15.43:0/2373065556 >> [v2:172.21.15.43:6835/1096784002,v1:172.21.15.43:6837/1096784002] conn(0x55fef75a4c80 0x55fef75a5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.950+0000 7f819c9b3b80 4 client.5277 mds_command: new command op to 5112 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-28T20:08:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.950+0000 7f819c9b3b80 1 -- 172.21.15.43:0/2373065556 --> [v2:172.21.15.43:6835/1096784002,v1:172.21.15.43:6837/1096784002] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fef74cce60 con 0x55fef75a4c80 2023-06-28T20:08:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.950+0000 7f818a59c700 1 --2- 172.21.15.43:0/2373065556 >> [v2:172.21.15.43:6835/1096784002,v1:172.21.15.43:6837/1096784002] conn(0x55fef75a4c80 0x55fef75a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.951+0000 7f81697fa700 10 client.5277 ms_handle_connect on v2:172.21.15.43:6835/1096784002 2023-06-28T20:08:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.951+0000 7f818a59c700 1 --2- 172.21.15.43:0/2373065556 >> [v2:172.21.15.43:6835/1096784002,v1:172.21.15.43:6837/1096784002] conn(0x55fef75a4c80 0x55fef75a5050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=942d81d41e9966d1 server_cookie=d8630ebd3d30a9ba in_seq=0 out_seq=0 2023-06-28T20:08:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.951+0000 7f81697fa700 1 -- 172.21.15.43:0/2373065556 <== mds.0 v2:172.21.15.43:6835/1096784002 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fef74cce60 con 0x55fef75a4c80 2023-06-28T20:08:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.951+0000 7f81697fa700 10 client.5277 handle_command_reply: tid=0 2023-06-28T20:08:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.951+0000 7f819c9b3b80 1 client.5277 shutdown 2023-06-28T20:08:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.951+0000 7f819c9b3b80 1 -- 172.21.15.43:0/2373065556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef75a2170 msgr2=0x55fef75c0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.951+0000 7f819c9b3b80 1 --2- 172.21.15.43:0/2373065556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef75a2170 0x55fef75c0060 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f817c013000 tx=0x7f817c0047c0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.951+0000 7f819c9b3b80 1 -- 172.21.15.43:0/2373065556 shutdown_connections 2023-06-28T20:08:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.951+0000 7f819c9b3b80 1 --2- 172.21.15.43:0/2373065556 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fef75a2540 0x55fef75c05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.951+0000 7f819c9b3b80 1 --2- 172.21.15.43:0/2373065556 >> [v2:172.21.15.43:6835/1096784002,v1:172.21.15.43:6837/1096784002] conn(0x55fef75a4c80 0x55fef75a5050 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.951+0000 7f819c9b3b80 1 --2- 172.21.15.43:0/2373065556 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fef757d6c0 0x55fef75a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.951+0000 7f81697fa700 0 client.5277 ms_handle_reset on v2:172.21.15.43:6835/1096784002 2023-06-28T20:08:26.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.951+0000 7f819c9b3b80 1 --2- 172.21.15.43:0/2373065556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef75a2170 0x55fef75c0060 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.951+0000 7f819c9b3b80 1 -- 172.21.15.43:0/2373065556 >> 172.21.15.43:0/2373065556 conn(0x55fef7577a30 msgr2=0x55fef75cb320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:26.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.952+0000 7f819c9b3b80 1 -- 172.21.15.43:0/2373065556 shutdown_connections 2023-06-28T20:08:26.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.952+0000 7f819c9b3b80 1 -- 172.21.15.43:0/2373065556 wait complete. 2023-06-28T20:08:26.977 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.952+0000 7f819c9b3b80 20 client.5277 trim_cache size 0 max 16384 2023-06-28T20:08:26.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.970+0000 7f819c9b3b80 1 Processor -- start 2023-06-28T20:08:26.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.970+0000 7f819c9b3b80 1 -- start start 2023-06-28T20:08:26.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.970+0000 7f819c9b3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fef757d6c0 0x55fef75843d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.970+0000 7f819c9b3b80 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef75a2170 0x55fef75c0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.982 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.970+0000 7f819c9b3b80 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fef75a2540 0x55fef75c0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.970+0000 7f819c9b3b80 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fef75e9350 con 0x55fef75a2170 2023-06-28T20:08:26.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.970+0000 7f819c9b3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fef75bf3e0 con 0x55fef757d6c0 2023-06-28T20:08:26.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.970+0000 7f819c9b3b80 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fef75a9170 con 0x55fef75a2540 2023-06-28T20:08:26.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.970+0000 7f8189d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef75a2170 0x55fef75c0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:26.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.970+0000 7f8189d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef75a2170 0x55fef75c0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55132/0 (socket says 172.21.15.43:55132) 2023-06-28T20:08:26.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.970+0000 7f8189d9b700 1 -- 172.21.15.43:0/1304926479 learned_addr learned my addr 172.21.15.43:0/1304926479 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:26.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.971+0000 7f8189d9b700 1 -- 172.21.15.43:0/1304926479 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fef75a2540 msgr2=0x55fef75c0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:26.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.971+0000 7f818ad9d700 1 --2- 172.21.15.43:0/1304926479 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fef75a2540 0x55fef75c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:26.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.971+0000 7f8189d9b700 1 --2- 172.21.15.43:0/1304926479 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fef75a2540 0x55fef75c0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.971+0000 7f8189d9b700 1 -- 172.21.15.43:0/1304926479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fef757d6c0 msgr2=0x55fef75843d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:26.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.971+0000 7f8189d9b700 1 --2- 172.21.15.43:0/1304926479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fef757d6c0 0x55fef75843d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.971+0000 7f8189d9b700 1 -- 172.21.15.43:0/1304926479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fef75cc450 con 0x55fef75a2170 2023-06-28T20:08:26.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.971+0000 7f8189d9b700 1 --2- 172.21.15.43:0/1304926479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef75a2170 0x55fef75c0040 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f817c014040 tx=0x7f817c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ec3e9710530c6a server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:26.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.971+0000 7f8178ff9700 1 -- 172.21.15.43:0/1304926479 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c02e430 con 0x55fef75a2170 2023-06-28T20:08:26.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.971+0000 7f8178ff9700 1 -- 172.21.15.43:0/1304926479 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f817c02e590 con 0x55fef75a2170 2023-06-28T20:08:26.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.971+0000 7f8178ff9700 1 -- 172.21.15.43:0/1304926479 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c0038a0 con 0x55fef75a2170 2023-06-28T20:08:26.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.971+0000 7f819c9b3b80 1 -- 172.21.15.43:0/1304926479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef75a2170 msgr2=0x55fef75c0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:26.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.971+0000 7f819c9b3b80 1 --2- 172.21.15.43:0/1304926479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef75a2170 0x55fef75c0040 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f817c014040 tx=0x7f817c022000 comp rx=0 tx=0).stop 2023-06-28T20:08:26.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.972+0000 7f819c9b3b80 1 -- 172.21.15.43:0/1304926479 shutdown_connections 2023-06-28T20:08:26.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.972+0000 7f819c9b3b80 1 --2- 172.21.15.43:0/1304926479 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fef75a2540 0x55fef75c0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.972+0000 7f819c9b3b80 1 --2- 172.21.15.43:0/1304926479 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fef757d6c0 0x55fef75843d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.972+0000 7f819c9b3b80 1 --2- 172.21.15.43:0/1304926479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef75a2170 0x55fef75c0040 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.972+0000 7f819c9b3b80 1 -- 172.21.15.43:0/1304926479 >> 172.21.15.43:0/1304926479 conn(0x55fef7577a30 msgr2=0x55fef75d6800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:26.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.972+0000 7f819c9b3b80 1 -- 172.21.15.43:0/1304926479 shutdown_connections 2023-06-28T20:08:26.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.972+0000 7f819c9b3b80 1 -- 172.21.15.43:0/1304926479 wait complete. 2023-06-28T20:08:26.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.972+0000 7f819c9b3b80 1 -- start start 2023-06-28T20:08:26.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.972+0000 7f819c9b3b80 1 Processor -- start 2023-06-28T20:08:26.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.974+0000 7f819c9b3b80 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef757d6c0 0x55fef75c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.974+0000 7f819c9b3b80 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fef75a2170 0x55fef75c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.974+0000 7f819c9b3b80 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fef75a2540 0x55fef75ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.974+0000 7f819c9b3b80 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fef75c9a00 con 0x55fef757d6c0 2023-06-28T20:08:26.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.974+0000 7f819c9b3b80 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fef75bd960 con 0x55fef75a2540 2023-06-28T20:08:26.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.974+0000 7f819c9b3b80 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fef75a87e0 con 0x55fef75a2170 2023-06-28T20:08:26.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.974+0000 7f819c9b3b80 10 client.0 _ll_register_callbacks cb 0x7f817c009380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-28T20:08:26.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.974+0000 7f818a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef757d6c0 0x55fef75c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:26.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.974+0000 7f818a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef757d6c0 0x55fef75c76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55136/0 (socket says 172.21.15.43:55136) 2023-06-28T20:08:26.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.974+0000 7f818a59c700 1 -- 172.21.15.43:0/679438076 learned_addr learned my addr 172.21.15.43:0/679438076 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:26.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.974+0000 7f818a59c700 1 -- 172.21.15.43:0/679438076 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fef75a2170 msgr2=0x55fef75c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:26.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.974+0000 7f818a59c700 1 --2- 172.21.15.43:0/679438076 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fef75a2170 0x55fef75c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.974+0000 7f818a59c700 1 -- 172.21.15.43:0/679438076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fef75a2540 msgr2=0x55fef75ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:26.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.974+0000 7f818a59c700 1 --2- 172.21.15.43:0/679438076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fef75a2540 0x55fef75ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.974+0000 7f818a59c700 1 -- 172.21.15.43:0/679438076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fef72eac10 con 0x55fef757d6c0 2023-06-28T20:08:26.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.974+0000 7f818a59c700 1 --2- 172.21.15.43:0/679438076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef757d6c0 0x55fef75c76e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f818c156710 tx=0x7f8180000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19dd7243cbaf37f server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:26.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.974+0000 7f81697fa700 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2023-06-28T20:08:26.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.974+0000 7f81697fa700 1 -- 172.21.15.43:0/679438076 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180025a70 con 0x55fef757d6c0 2023-06-28T20:08:26.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.974+0000 7f81697fa700 1 -- 172.21.15.43:0/679438076 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8180025bd0 con 0x55fef757d6c0 2023-06-28T20:08:26.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.974+0000 7f819c9b3b80 1 -- 172.21.15.43:0/679438076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fef748f000 con 0x55fef757d6c0 2023-06-28T20:08:26.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.975+0000 7f81697fa700 1 -- 172.21.15.43:0/679438076 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8180025d90 con 0x55fef757d6c0 2023-06-28T20:08:26.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.975+0000 7f819c9b3b80 10 client.5283 fetch_fsmap learned FSMap version 15 2023-06-28T20:08:26.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.975+0000 7f819c9b3b80 1 -- 172.21.15.43:0/679438076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x55fef7328200 con 0x55fef757d6c0 2023-06-28T20:08:26.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.975+0000 7f81697fa700 1 -- 172.21.15.43:0/679438076 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fef748f000 con 0x55fef757d6c0 2023-06-28T20:08:26.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.975+0000 7f81697fa700 1 -- 172.21.15.43:0/679438076 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 15) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f818002c060 con 0x55fef757d6c0 2023-06-28T20:08:26.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.975+0000 7f819c9b3b80 10 client.5283 fetch_fsmap finished waiting for FSMap version 15 2023-06-28T20:08:26.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.975+0000 7f819c9b3b80 10 client.5283 resolve_mds: resolved name 'a' to daemon mds.a 2023-06-28T20:08:26.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.975+0000 7f819c9b3b80 20 client.5283 populate_metadata read hostname 'smithi043' 2023-06-28T20:08:26.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.975+0000 7f819c9b3b80 1 --2- 172.21.15.43:0/679438076 >> [v2:172.21.15.43:6835/1096784002,v1:172.21.15.43:6837/1096784002] conn(0x55fef75aeff0 0x55fef75af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:26.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.975+0000 7f819c9b3b80 4 client.5283 mds_command: new command op to 5112 tid=0[{"prefix": "status"}] 2023-06-28T20:08:26.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.975+0000 7f819c9b3b80 1 -- 172.21.15.43:0/679438076 --> [v2:172.21.15.43:6835/1096784002,v1:172.21.15.43:6837/1096784002] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fef7322f20 con 0x55fef75aeff0 2023-06-28T20:08:26.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.975+0000 7f8189d9b700 1 --2- 172.21.15.43:0/679438076 >> [v2:172.21.15.43:6835/1096784002,v1:172.21.15.43:6837/1096784002] conn(0x55fef75aeff0 0x55fef75af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:26.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.975+0000 7f8189d9b700 1 --2- 172.21.15.43:0/679438076 >> [v2:172.21.15.43:6835/1096784002,v1:172.21.15.43:6837/1096784002] conn(0x55fef75aeff0 0x55fef75af3c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=224cb31a29d2b9ff server_cookie=1a6c7dc3a7b8036a in_seq=0 out_seq=0 2023-06-28T20:08:26.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.975+0000 7f81697fa700 10 client.5283 ms_handle_connect on v2:172.21.15.43:6835/1096784002 2023-06-28T20:08:26.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f81697fa700 1 -- 172.21.15.43:0/679438076 <== mds.0 v2:172.21.15.43:6835/1096784002 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55fef7322f20 con 0x55fef75aeff0 2023-06-28T20:08:26.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f81697fa700 10 client.5283 handle_command_reply: tid=0 2023-06-28T20:08:26.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f819c9b3b80 1 client.5283 shutdown 2023-06-28T20:08:26.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f819c9b3b80 1 -- 172.21.15.43:0/679438076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef757d6c0 msgr2=0x55fef75c76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:26.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f819c9b3b80 1 --2- 172.21.15.43:0/679438076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef757d6c0 0x55fef75c76e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f818c156710 tx=0x7f8180000ed0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f819c9b3b80 1 -- 172.21.15.43:0/679438076 shutdown_connections 2023-06-28T20:08:26.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f819c9b3b80 1 --2- 172.21.15.43:0/679438076 >> [v2:172.21.15.43:6835/1096784002,v1:172.21.15.43:6837/1096784002] conn(0x55fef75aeff0 0x55fef75af3c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f819c9b3b80 1 --2- 172.21.15.43:0/679438076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x55fef75a2540 0x55fef75ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f81697fa700 0 client.5283 ms_handle_reset on v2:172.21.15.43:6835/1096784002 2023-06-28T20:08:26.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f819c9b3b80 1 --2- 172.21.15.43:0/679438076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fef757d6c0 0x55fef75c76e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f819c9b3b80 1 --2- 172.21.15.43:0/679438076 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x55fef75a2170 0x55fef75c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f819c9b3b80 1 -- 172.21.15.43:0/679438076 >> 172.21.15.43:0/679438076 conn(0x55fef7577a30 msgr2=0x55fef75e8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:26.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f819c9b3b80 1 -- 172.21.15.43:0/679438076 shutdown_connections 2023-06-28T20:08:26.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f819c9b3b80 1 -- 172.21.15.43:0/679438076 wait complete. 2023-06-28T20:08:26.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f819c9b3b80 20 client.5283 trim_cache size 0 max 16384 2023-06-28T20:08:26.993 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-06-28T20:08:26.993 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "b8fd740d-00b9-4308-92a8-48f5fffe8078", 2023-06-28T20:08:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2023-06-28T20:08:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 5112, 2023-06-28T20:08:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2023-06-28T20:08:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2023-06-28T20:08:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2023-06-28T20:08:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 6.3219283390000003, 2023-06-28T20:08:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 15, 2023-06-28T20:08:26.994 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 35, 2023-06-28T20:08:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 35, 2023-06-28T20:08:26.995 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 15.184136985 2023-06-28T20:08:26.995 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-06-28T20:08:26.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f819082f700 1 -- 172.21.15.43:0/94319426 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f816c06ace0 msgr2=0x7f816c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:26.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f819082f700 1 --2- 172.21.15.43:0/94319426 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f816c06ace0 0x7f816c06d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8180007fd0 tx=0x7f8180007de0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f819082f700 1 -- 172.21.15.43:0/94319426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f818c0af060 msgr2=0x7f818c1558c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:26.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f819082f700 1 --2- 172.21.15.43:0/94319426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f818c0af060 0x7f818c1558c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f818400cef0 tx=0x7f818400cf20 comp rx=0 tx=0).stop 2023-06-28T20:08:26.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f819082f700 1 -- 172.21.15.43:0/94319426 shutdown_connections 2023-06-28T20:08:26.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f819082f700 1 --2- 172.21.15.43:0/94319426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f818c0af060 0x7f818c1558c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f819082f700 1 --2- 172.21.15.43:0/94319426 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f816c06ace0 0x7f816c06d190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f819082f700 1 --2- 172.21.15.43:0/94319426 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f818c06ccc0 0x7f818c1560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f819082f700 1 --2- 172.21.15.43:0/94319426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f818c0aec90 0x7f818c15d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:26.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.976+0000 7f819082f700 1 -- 172.21.15.43:0/94319426 >> 172.21.15.43:0/94319426 conn(0x7f818c06b000 msgr2=0x7f818c158250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:26.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.977+0000 7f819082f700 1 -- 172.21.15.43:0/94319426 shutdown_connections 2023-06-28T20:08:26.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:26.977+0000 7f819082f700 1 -- 172.21.15.43:0/94319426 wait complete. 2023-06-28T20:08:26.998 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-28T20:08:27.123 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.119+0000 7ff4f3a58700 1 Processor -- start 2023-06-28T20:08:27.124 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.119+0000 7ff4f3a58700 1 -- start start 2023-06-28T20:08:27.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.120+0000 7ff4f3a58700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4ec0b2170 0x7ff4ec0d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:27.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.120+0000 7ff4f3a58700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff4ec0cf120 0x7ff4ec0b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:27.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.120+0000 7ff4f3a58700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4ec160270 con 0x7ff4ec0d2b00 2023-06-28T20:08:27.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.120+0000 7ff4f3a58700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4ec0d0910 con 0x7ff4ec0cf120 2023-06-28T20:08:27.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.120+0000 7ff4f3a58700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec0cf870 con 0x7ff4ec0b2170 2023-06-28T20:08:27.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.120+0000 7ff4f0ff3700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff4ec0cf120 0x7ff4ec0b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:27.126 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.120+0000 7ff4f1ff5700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff4ec0d2b00 0x7ff4ec06a7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44334/0 (socket says 172.21.15.43:44334) 2023-06-28T20:08:27.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.120+0000 7ff4f1ff5700 1 -- 172.21.15.43:0/2033017185 learned_addr learned my addr 172.21.15.43:0/2033017185 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:27.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.121+0000 7ff4f17f4700 1 --2- 172.21.15.43:0/2033017185 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4ec0b2170 0x7ff4ec0d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:27.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.121+0000 7ff4f0ff3700 1 -- 172.21.15.43:0/2033017185 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4ec0b2170 msgr2=0x7ff4ec0d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:27.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.121+0000 7ff4f0ff3700 1 --2- 172.21.15.43:0/2033017185 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4ec0b2170 0x7ff4ec0d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:27.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.121+0000 7ff4f0ff3700 1 -- 172.21.15.43:0/2033017185 >> v1:172.21.15.43:6789/0 conn(0x7ff4ec0d2b00 legacy=0x7ff4ec06a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:27.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.121+0000 7ff4f0ff3700 1 -- 172.21.15.43:0/2033017185 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ec0cf590 con 0x7ff4ec0cf120 2023-06-28T20:08:27.127 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.121+0000 7ff4e3fff700 1 -- 172.21.15.43:0/2033017185 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1980696595 0 0) 0x7ff4ec160270 con 0x7ff4ec0d2b00 2023-06-28T20:08:27.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.121+0000 7ff4f17f4700 1 --2- 172.21.15.43:0/2033017185 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4ec0b2170 0x7ff4ec0d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:27.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.121+0000 7ff4f0ff3700 1 --2- 172.21.15.43:0/2033017185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4ec0cf120 0x7ff4ec0b5e30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009870 tx=0x7ff4e8009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf2be48eb8b0c7b server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:27.128 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.121+0000 7ff4e3fff700 1 -- 172.21.15.43:0/2033017185 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8011830 con 0x7ff4ec0cf120 2023-06-28T20:08:27.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.121+0000 7ff4e3fff700 1 -- 172.21.15.43:0/2033017185 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e8011990 con 0x7ff4ec0cf120 2023-06-28T20:08:27.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.121+0000 7ff4e3fff700 1 -- 172.21.15.43:0/2033017185 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8011d20 con 0x7ff4ec0cf120 2023-06-28T20:08:27.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.122+0000 7ff4f3a58700 1 -- 172.21.15.43:0/2033017185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4ec0cf120 msgr2=0x7ff4ec0b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:27.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.122+0000 7ff4f3a58700 1 --2- 172.21.15.43:0/2033017185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4ec0cf120 0x7ff4ec0b5e30 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009870 tx=0x7ff4e8009b80 comp rx=0 tx=0).stop 2023-06-28T20:08:27.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.122+0000 7ff4f3a58700 1 -- 172.21.15.43:0/2033017185 shutdown_connections 2023-06-28T20:08:27.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.122+0000 7ff4f3a58700 1 --2- 172.21.15.43:0/2033017185 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4ec0b2170 0x7ff4ec0d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:27.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.122+0000 7ff4f3a58700 1 --2- 172.21.15.43:0/2033017185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4ec0cf120 0x7ff4ec0b5e30 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:27.129 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.122+0000 7ff4f3a58700 1 -- 172.21.15.43:0/2033017185 >> 172.21.15.43:0/2033017185 conn(0x7ff4ec06c1e0 msgr2=0x7ff4ec06c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:27.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.122+0000 7ff4f3a58700 1 -- 172.21.15.43:0/2033017185 shutdown_connections 2023-06-28T20:08:27.130 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.122+0000 7ff4f3a58700 1 -- 172.21.15.43:0/2033017185 wait complete. 2023-06-28T20:08:27.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.123+0000 7ff4f3a58700 1 Processor -- start 2023-06-28T20:08:27.132 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.123+0000 7ff4f3a58700 1 -- start start 2023-06-28T20:08:27.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.124+0000 7ff4f3a58700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4ec0b2170 0x7ff4ec158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:27.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.124+0000 7ff4f3a58700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4ec0cf120 0x7ff4ec15d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:27.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.124+0000 7ff4f3a58700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4ec0b8940 0x7ff4ec15f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:27.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.124+0000 7ff4f3a58700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec15e4d0 con 0x7ff4ec0cf120 2023-06-28T20:08:27.133 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.124+0000 7ff4f3a58700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec15ed70 con 0x7ff4ec0b8940 2023-06-28T20:08:27.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.124+0000 7ff4f3a58700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec1585c0 con 0x7ff4ec0b2170 2023-06-28T20:08:27.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.124+0000 7ff4f0ff3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4ec0cf120 0x7ff4ec15d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:27.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.124+0000 7ff4f0ff3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4ec0cf120 0x7ff4ec15d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55152/0 (socket says 172.21.15.43:55152) 2023-06-28T20:08:27.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.124+0000 7ff4f0ff3700 1 -- 172.21.15.43:0/3888507095 learned_addr learned my addr 172.21.15.43:0/3888507095 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:27.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.124+0000 7ff4f17f4700 1 --2- 172.21.15.43:0/3888507095 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4ec0b2170 0x7ff4ec158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:27.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.124+0000 7ff4f1ff5700 1 --2- 172.21.15.43:0/3888507095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4ec0b8940 0x7ff4ec15f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:27.134 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.124+0000 7ff4f0ff3700 1 -- 172.21.15.43:0/3888507095 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4ec0b2170 msgr2=0x7ff4ec158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:27.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.124+0000 7ff4f0ff3700 1 --2- 172.21.15.43:0/3888507095 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4ec0b2170 0x7ff4ec158080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:27.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.124+0000 7ff4f0ff3700 1 -- 172.21.15.43:0/3888507095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4ec0b8940 msgr2=0x7ff4ec15f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:27.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.124+0000 7ff4f0ff3700 1 --2- 172.21.15.43:0/3888507095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4ec0b8940 0x7ff4ec15f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:27.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.124+0000 7ff4f0ff3700 1 -- 172.21.15.43:0/3888507095 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e80094a0 con 0x7ff4ec0cf120 2023-06-28T20:08:27.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.124+0000 7ff4f1ff5700 1 --2- 172.21.15.43:0/3888507095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4ec0b8940 0x7ff4ec15f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:27.135 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.124+0000 7ff4f0ff3700 1 --2- 172.21.15.43:0/3888507095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4ec0cf120 0x7ff4ec15d720 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80097c0 tx=0x7ff4e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:27.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.125+0000 7ff4e1ffb700 1 -- 172.21.15.43:0/3888507095 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e80045f0 con 0x7ff4ec0cf120 2023-06-28T20:08:27.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.125+0000 7ff4e1ffb700 1 -- 172.21.15.43:0/3888507095 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e801f070 con 0x7ff4ec0cf120 2023-06-28T20:08:27.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.125+0000 7ff4f3a58700 1 -- 172.21.15.43:0/3888507095 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4ec15df70 con 0x7ff4ec0cf120 2023-06-28T20:08:27.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.125+0000 7ff4e1ffb700 1 -- 172.21.15.43:0/3888507095 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8003ae0 con 0x7ff4ec0cf120 2023-06-28T20:08:27.136 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.125+0000 7ff4f3a58700 1 -- 172.21.15.43:0/3888507095 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4ec15ea00 con 0x7ff4ec0cf120 2023-06-28T20:08:27.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.126+0000 7ff4f3a58700 1 -- 172.21.15.43:0/3888507095 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4ec0728b0 con 0x7ff4ec0cf120 2023-06-28T20:08:27.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.126+0000 7ff4e1ffb700 1 -- 172.21.15.43:0/3888507095 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff4e8003c40 con 0x7ff4ec0cf120 2023-06-28T20:08:27.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.127+0000 7ff4e1ffb700 1 --2- 172.21.15.43:0/3888507095 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff4d006ace0 0x7ff4d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:27.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.127+0000 7ff4e1ffb700 1 -- 172.21.15.43:0/3888507095 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff4e8021070 con 0x7ff4ec0cf120 2023-06-28T20:08:27.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.127+0000 7ff4f17f4700 1 --2- 172.21.15.43:0/3888507095 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff4d006ace0 0x7ff4d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:27.137 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.127+0000 7ff4f17f4700 1 --2- 172.21.15.43:0/3888507095 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff4d006ace0 0x7ff4d006d190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc006fd0 tx=0x7ff4dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:27.138 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.131+0000 7ff4e1ffb700 1 -- 172.21.15.43:0/3888507095 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff4e8056090 con 0x7ff4ec0cf120 2023-06-28T20:08:27.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.378+0000 7ff4f3a58700 1 -- 172.21.15.43:0/3888507095 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff4ec15eeb0 con 0x7ff4ec0cf120 2023-06-28T20:08:27.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.378+0000 7ff4e1ffb700 1 -- 172.21.15.43:0/3888507095 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff4e8015070 con 0x7ff4ec0cf120 2023-06-28T20:08:27.382 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-06-28T20:08:27.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.380+0000 7ff4f3a58700 1 -- 172.21.15.43:0/3888507095 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff4d006ace0 msgr2=0x7ff4d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.380+0000 7ff4f3a58700 1 --2- 172.21.15.43:0/3888507095 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff4d006ace0 0x7ff4d006d190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc006fd0 tx=0x7ff4dc008040 comp rx=0 tx=0).stop 2023-06-28T20:08:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.380+0000 7ff4f3a58700 1 -- 172.21.15.43:0/3888507095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4ec0cf120 msgr2=0x7ff4ec15d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.380+0000 7ff4f3a58700 1 --2- 172.21.15.43:0/3888507095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4ec0cf120 0x7ff4ec15d720 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80097c0 tx=0x7ff4e8004750 comp rx=0 tx=0).stop 2023-06-28T20:08:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.380+0000 7ff4f3a58700 1 -- 172.21.15.43:0/3888507095 shutdown_connections 2023-06-28T20:08:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.380+0000 7ff4f3a58700 1 --2- 172.21.15.43:0/3888507095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff4ec0b8940 0x7ff4ec15f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.380+0000 7ff4f3a58700 1 --2- 172.21.15.43:0/3888507095 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff4d006ace0 0x7ff4d006d190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:27.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.380+0000 7ff4f3a58700 1 --2- 172.21.15.43:0/3888507095 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff4ec0b2170 0x7ff4ec158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.380+0000 7ff4f3a58700 1 --2- 172.21.15.43:0/3888507095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4ec0cf120 0x7ff4ec15d720 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.380+0000 7ff4f3a58700 1 -- 172.21.15.43:0/3888507095 >> 172.21.15.43:0/3888507095 conn(0x7ff4ec06c1e0 msgr2=0x7ff4ec160700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.380+0000 7ff4f3a58700 1 -- 172.21.15.43:0/3888507095 shutdown_connections 2023-06-28T20:08:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:27.380+0000 7ff4f3a58700 1 -- 172.21.15.43:0/3888507095 wait complete. 2023-06-28T20:08:27.397 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-06-28T20:08:27.397 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-28T20:08:27.398 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2023-06-28T20:08:27.398 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-06-28T20:08:27.398 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-06-28T20:08:27.398 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-06-28T20:08:27.399 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-06-28T20:08:27.399 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-28T20:08:27.399 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-06-28T20:08:27.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:27.400 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2023-06-28T20:08:27.418 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-28T20:08:27.418 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-28T20:08:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2023-06-28T20:08:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:08:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2023-06-28T20:08:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:08:27.419 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-28T20:08:27.419 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2023-06-28T20:08:27.419 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-28T20:08:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2023-06-28T20:08:27.420 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-28T20:08:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2023-06-28T20:08:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-06-28T20:08:27.420 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 41852sec preferred_lft 41852sec 2023-06-28T20:08:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2023-06-28T20:08:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:08:27.421 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-28T20:08:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2023-06-28T20:08:27.421 INFO:teuthology.orchestra.run.smithi043.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-28T20:08:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 06:bd:a8:fa:4b:47 brd ff:ff:ff:ff:ff:ff 2023-06-28T20:08:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-06-28T20:08:27.421 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:08:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::86f:f5ff:fef2:e36f/64 scope link 2023-06-28T20:08:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:08:27.422 INFO:teuthology.orchestra.run.smithi043.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-28T20:08:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 06:bd:a8:fa:4b:47 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-06-28T20:08:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::4bd:a8ff:fefa:4b47/64 scope link 2023-06-28T20:08:27.422 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:08:27.422 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-28T20:08:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 62:19:bc:9f:01:f4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-06-28T20:08:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::6019:bcff:fe9f:1f4/64 scope link 2023-06-28T20:08:27.423 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:08:27.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:27.423 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-28T20:08:27.485 INFO:teuthology.orchestra.run.smithi043.stdout:1 2023-06-28T20:08:27.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:27.486 DEBUG:teuthology.orchestra.run.smithi043:> route 2023-06-28T20:08:27.546 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2023-06-28T20:08:27.547 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-28T20:08:27.547 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-06-28T20:08:27.547 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-06-28T20:08:27.547 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-28T20:08:27.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:27.548 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2023-06-28T20:08:27.548 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2023-06-28T20:08:27.549 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-06-28T20:08:27.549 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-06-28T20:08:27.549 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-06-28T20:08:27.550 DEBUG:teuthology.orchestra.run.smithi043:> ') 2023-06-28T20:08:27.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:27.654 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-06-28T20:08:27.672 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-06-28T20:08:27.672 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-06-28T20:08:27.672 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-06-28T20:08:27.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:27.673 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2023-06-28T20:08:27.740 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-28T20:08:27.740 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-06-28T20:08:27.797 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-06-28T20:08:27.797 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-28T20:08:27.798 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-06-28T20:08:27.853 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-06-28T20:08:27.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:27.854 DEBUG:teuthology.orchestra.run.smithi043:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-06-28T20:08:27.940 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-06-28T20:08:27.940 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-06-28T20:08:27.940 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2023-06-28T20:08:27.949 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-06-28T20:08:27.949 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-06-28T20:08:27.949 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2023-06-28T20:08:27.949 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.178:3300,172.21.15.178:3301" 2023-06-28T20:08:27.950 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2023-06-28T20:08:27.985 INFO:teuthology.orchestra.run.smithi043.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-06-28T20:08:27.985 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-06-28T20:08:27.985 INFO:teuthology.orchestra.run.smithi043.stdout:invalid new device string format 2023-06-28T20:08:27.985 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.178:3300,172.21.15.178:3301" 2023-06-28T20:08:27.986 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: switching to using v1 address with old syntax 2023-06-28T20:08:27.986 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.178:6789,172.21.15.178:6790" 2023-06-28T20:08:27.986 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2023-06-28T20:08:27.986 INFO:teuthology.orchestra.run.smithi043.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=b8fd740d-00b9-4308-92a8-48f5fffe8078" will pass to kernel 2023-06-28T20:08:27.991 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-06-28T20:08:27.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:27.992 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-06-28T20:08:28.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:28.029 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-28T20:08:28.110 INFO:teuthology.orchestra.run.smithi043.stdout:client5295 2023-06-28T20:08:28.110 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:08:28.111 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:08:28.168 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-28T20:08:28.332 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.328+0000 7f509863c700 1 Processor -- start 2023-06-28T20:08:28.333 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.328+0000 7f509863c700 1 -- start start 2023-06-28T20:08:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.329+0000 7f509863c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5090157060 0x7f509015a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.329+0000 7f509863c700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5090157430 0x7f50900cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:28.335 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.329+0000 7f509863c700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f509015d720 con 0x7f509015d290 2023-06-28T20:08:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.329+0000 7f509863c700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50900ccf40 con 0x7f5090157430 2023-06-28T20:08:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.329+0000 7f509863c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509015da00 con 0x7f5090157060 2023-06-28T20:08:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.329+0000 7f5096bd9700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5090157430 0x7f50900cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.329+0000 7f50963d8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5090157060 0x7f509015a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.330+0000 7f5096bd9700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5090157430 0x7f50900cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:41226/0 (socket says 172.21.15.43:41226) 2023-06-28T20:08:28.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.330+0000 7f5096bd9700 1 -- 172.21.15.43:0/3303006282 learned_addr learned my addr 172.21.15.43:0/3303006282 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:28.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.330+0000 7f5095bd7700 1 --1- 172.21.15.43:0/3303006282 >> v1:172.21.15.178:6789/0 conn(0x7f509015d290 0x7f50900cea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:43472/0 (socket says 172.21.15.43:43472) 2023-06-28T20:08:28.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.330+0000 7f50963d8700 1 -- 172.21.15.43:0/3303006282 >> v2:172.21.15.178:3300/0 conn(0x7f5090157430 msgr2=0x7f50900cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:28.337 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.330+0000 7f50963d8700 1 --2- 172.21.15.43:0/3303006282 >> v2:172.21.15.178:3300/0 conn(0x7f5090157430 0x7f50900cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:28.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.330+0000 7f50963d8700 1 -- 172.21.15.43:0/3303006282 >> v1:172.21.15.178:6789/0 conn(0x7f509015d290 legacy=0x7f50900cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:28.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.330+0000 7f50953d6700 1 -- 172.21.15.43:0/3303006282 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 156976431 0 0) 0x7f509015d720 con 0x7f509015d290 2023-06-28T20:08:28.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.330+0000 7f50963d8700 1 -- 172.21.15.43:0/3303006282 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509015dce0 con 0x7f5090157060 2023-06-28T20:08:28.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.330+0000 7f5096bd9700 1 --2- 172.21.15.43:0/3303006282 >> v2:172.21.15.178:3300/0 conn(0x7f5090157430 0x7f50900cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:08:28.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.330+0000 7f50963d8700 1 --2- 172.21.15.43:0/3303006282 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5090157060 0x7f509015a720 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5080009840 tx=0x7f5080009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc51afa33c803e1f server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:28.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.330+0000 7f50953d6700 1 -- 172.21.15.43:0/3303006282 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5080011830 con 0x7f5090157060 2023-06-28T20:08:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.330+0000 7f50953d6700 1 -- 172.21.15.43:0/3303006282 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5080011990 con 0x7f5090157060 2023-06-28T20:08:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.330+0000 7f50953d6700 1 -- 172.21.15.43:0/3303006282 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5080011d20 con 0x7f5090157060 2023-06-28T20:08:28.340 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.331+0000 7f509863c700 1 -- 172.21.15.43:0/3303006282 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5090157060 msgr2=0x7f509015a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:28.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.331+0000 7f509863c700 1 --2- 172.21.15.43:0/3303006282 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5090157060 0x7f509015a720 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5080009840 tx=0x7f5080009b50 comp rx=0 tx=0).stop 2023-06-28T20:08:28.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.331+0000 7f509863c700 1 -- 172.21.15.43:0/3303006282 shutdown_connections 2023-06-28T20:08:28.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.331+0000 7f509863c700 1 --2- 172.21.15.43:0/3303006282 >> v2:172.21.15.178:3300/0 conn(0x7f5090157430 0x7f50900cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:28.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.331+0000 7f509863c700 1 --2- 172.21.15.43:0/3303006282 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5090157060 0x7f509015a720 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:28.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.331+0000 7f509863c700 1 -- 172.21.15.43:0/3303006282 >> 172.21.15.43:0/3303006282 conn(0x7f5090069ef0 msgr2=0x7f509006a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:28.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.331+0000 7f509863c700 1 -- 172.21.15.43:0/3303006282 shutdown_connections 2023-06-28T20:08:28.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.331+0000 7f509863c700 1 -- 172.21.15.43:0/3303006282 wait complete. 2023-06-28T20:08:28.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.332+0000 7f509863c700 1 Processor -- start 2023-06-28T20:08:28.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.332+0000 7f509863c700 1 -- start start 2023-06-28T20:08:28.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.333+0000 7f509863c700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5090157060 0x7f509015b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:28.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.333+0000 7f509863c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5090157430 0x7f50900c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:28.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.333+0000 7f509863c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f509015d290 0x7f50900c4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:28.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.333+0000 7f509863c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509014afa0 con 0x7f509015d290 2023-06-28T20:08:28.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.333+0000 7f509863c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50900c2e50 con 0x7f5090157430 2023-06-28T20:08:28.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.333+0000 7f509863c700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50900c3130 con 0x7f5090157060 2023-06-28T20:08:28.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.333+0000 7f5096bd9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f509015d290 0x7f50900c4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:28.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.333+0000 7f5096bd9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f509015d290 0x7f50900c4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55172/0 (socket says 172.21.15.43:55172) 2023-06-28T20:08:28.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.333+0000 7f5096bd9700 1 -- 172.21.15.43:0/2886978813 learned_addr learned my addr 172.21.15.43:0/2886978813 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:28.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.333+0000 7f5095bd7700 1 --2- 172.21.15.43:0/2886978813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5090157430 0x7f50900c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:28.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.333+0000 7f50963d8700 1 --2- 172.21.15.43:0/2886978813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5090157060 0x7f509015b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:28.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.334+0000 7f5096bd9700 1 -- 172.21.15.43:0/2886978813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5090157060 msgr2=0x7f509015b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:28.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.334+0000 7f5096bd9700 1 --2- 172.21.15.43:0/2886978813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5090157060 0x7f509015b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:28.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.334+0000 7f5096bd9700 1 -- 172.21.15.43:0/2886978813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5090157430 msgr2=0x7f50900c24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:28.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.334+0000 7f5096bd9700 1 --2- 172.21.15.43:0/2886978813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5090157430 0x7f50900c24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:28.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.334+0000 7f5096bd9700 1 -- 172.21.15.43:0/2886978813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5088009b30 con 0x7f509015d290 2023-06-28T20:08:28.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.334+0000 7f5095bd7700 1 --2- 172.21.15.43:0/2886978813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5090157430 0x7f50900c24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:28.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.334+0000 7f50963d8700 1 --2- 172.21.15.43:0/2886978813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5090157060 0x7f509015b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:28.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.334+0000 7f5096bd9700 1 --2- 172.21.15.43:0/2886978813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f509015d290 0x7f50900c4ae0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f508800ef90 tx=0x7f508800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:28.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.334+0000 7f5086ffd700 1 -- 172.21.15.43:0/2886978813 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5088018070 con 0x7f509015d290 2023-06-28T20:08:28.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.334+0000 7f5086ffd700 1 -- 172.21.15.43:0/2886978813 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508800f040 con 0x7f509015d290 2023-06-28T20:08:28.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.334+0000 7f509863c700 1 -- 172.21.15.43:0/2886978813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50800094a0 con 0x7f509015d290 2023-06-28T20:08:28.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.334+0000 7f5086ffd700 1 -- 172.21.15.43:0/2886978813 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5088003c40 con 0x7f509015d290 2023-06-28T20:08:28.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.334+0000 7f509863c700 1 -- 172.21.15.43:0/2886978813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f509015b580 con 0x7f509015d290 2023-06-28T20:08:28.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.335+0000 7f509863c700 1 -- 172.21.15.43:0/2886978813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f509015b740 con 0x7f509015d290 2023-06-28T20:08:28.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.336+0000 7f5086ffd700 1 -- 172.21.15.43:0/2886978813 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f508800c9a0 con 0x7f509015d290 2023-06-28T20:08:28.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.336+0000 7f5086ffd700 1 --2- 172.21.15.43:0/2886978813 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f507406ace0 0x7f507406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:28.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.336+0000 7f5086ffd700 1 -- 172.21.15.43:0/2886978813 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5088089660 con 0x7f509015d290 2023-06-28T20:08:28.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.336+0000 7f50963d8700 1 --2- 172.21.15.43:0/2886978813 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f507406ace0 0x7f507406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:28.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.336+0000 7f50963d8700 1 --2- 172.21.15.43:0/2886978813 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f507406ace0 0x7f507406d190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5080009fd0 tx=0x7f5080008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:28.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.340+0000 7f5086ffd700 1 -- 172.21.15.43:0/2886978813 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f50880547a0 con 0x7f509015d290 2023-06-28T20:08:28.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.588+0000 7f509863c700 1 -- 172.21.15.43:0/2886978813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f50900c3350 con 0x7f509015d290 2023-06-28T20:08:28.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.589+0000 7f5086ffd700 1 -- 172.21.15.43:0/2886978813 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5088057dc0 con 0x7f509015d290 2023-06-28T20:08:28.594 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-06-28T20:08:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.591+0000 7f509863c700 1 -- 172.21.15.43:0/2886978813 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f507406ace0 msgr2=0x7f507406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.591+0000 7f509863c700 1 --2- 172.21.15.43:0/2886978813 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f507406ace0 0x7f507406d190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5080009fd0 tx=0x7f5080008030 comp rx=0 tx=0).stop 2023-06-28T20:08:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.591+0000 7f509863c700 1 -- 172.21.15.43:0/2886978813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f509015d290 msgr2=0x7f50900c4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.591+0000 7f509863c700 1 --2- 172.21.15.43:0/2886978813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f509015d290 0x7f50900c4ae0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f508800ef90 tx=0x7f508800c3a0 comp rx=0 tx=0).stop 2023-06-28T20:08:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.591+0000 7f509863c700 1 -- 172.21.15.43:0/2886978813 shutdown_connections 2023-06-28T20:08:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.591+0000 7f509863c700 1 --2- 172.21.15.43:0/2886978813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f509015d290 0x7f50900c4ae0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.591+0000 7f509863c700 1 --2- 172.21.15.43:0/2886978813 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f507406ace0 0x7f507406d190 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.591+0000 7f509863c700 1 --2- 172.21.15.43:0/2886978813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5090157060 0x7f509015b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.591+0000 7f509863c700 1 --2- 172.21.15.43:0/2886978813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5090157430 0x7f50900c24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.591+0000 7f509863c700 1 -- 172.21.15.43:0/2886978813 >> 172.21.15.43:0/2886978813 conn(0x7f5090069ef0 msgr2=0x7f5090154c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.591+0000 7f509863c700 1 -- 172.21.15.43:0/2886978813 shutdown_connections 2023-06-28T20:08:28.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:28.591+0000 7f509863c700 1 -- 172.21.15.43:0/2886978813 wait complete. 2023-06-28T20:08:28.607 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-06-28T20:08:28.607 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-28T20:08:28.608 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi178.front.sepia.ceph.com 2023-06-28T20:08:28.608 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-06-28T20:08:28.608 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-06-28T20:08:28.608 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-06-28T20:08:28.609 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-06-28T20:08:28.609 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-28T20:08:28.609 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-06-28T20:08:28.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:28.613 DEBUG:teuthology.orchestra.run.smithi178:> ip addr 2023-06-28T20:08:28.635 INFO:teuthology.orchestra.run.smithi178.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-28T20:08:28.635 INFO:teuthology.orchestra.run.smithi178.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-28T20:08:28.636 INFO:teuthology.orchestra.run.smithi178.stdout: inet 127.0.0.1/8 scope host lo 2023-06-28T20:08:28.636 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:08:28.636 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 ::1/128 scope host 2023-06-28T20:08:28.636 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:08:28.636 INFO:teuthology.orchestra.run.smithi178.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-28T20:08:28.637 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:d9:3b:20 brd ff:ff:ff:ff:ff:ff 2023-06-28T20:08:28.637 INFO:teuthology.orchestra.run.smithi178.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-28T20:08:28.637 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:d9:3b:21 brd ff:ff:ff:ff:ff:ff 2023-06-28T20:08:28.637 INFO:teuthology.orchestra.run.smithi178.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-28T20:08:28.637 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:88:7e:f8 brd ff:ff:ff:ff:ff:ff 2023-06-28T20:08:28.637 INFO:teuthology.orchestra.run.smithi178.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-28T20:08:28.637 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:88:7e:f9 brd ff:ff:ff:ff:ff:ff 2023-06-28T20:08:28.638 INFO:teuthology.orchestra.run.smithi178.stdout: inet 172.21.15.178/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-06-28T20:08:28.638 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft 41908sec preferred_lft 41908sec 2023-06-28T20:08:28.638 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::6f8b:bfa0:af97:1715/64 scope link noprefixroute 2023-06-28T20:08:28.638 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:08:28.638 INFO:teuthology.orchestra.run.smithi178.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-06-28T20:08:28.638 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 12:59:7c:e9:62:f9 brd ff:ff:ff:ff:ff:ff 2023-06-28T20:08:28.638 INFO:teuthology.orchestra.run.smithi178.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-06-28T20:08:28.638 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:08:28.639 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::4031:96ff:fe27:8efd/64 scope link 2023-06-28T20:08:28.639 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:08:28.639 INFO:teuthology.orchestra.run.smithi178.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-28T20:08:28.639 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether ee:06:1c:61:d8:26 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-06-28T20:08:28.639 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::ec06:1cff:fe61:d826/64 scope link 2023-06-28T20:08:28.640 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:08:28.640 INFO:teuthology.orchestra.run.smithi178.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-06-28T20:08:28.640 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 12:59:7c:e9:62:f9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-06-28T20:08:28.640 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::1059:7cff:fee9:62f9/64 scope link 2023-06-28T20:08:28.641 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2023-06-28T20:08:28.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:28.642 DEBUG:teuthology.orchestra.run.smithi178:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-28T20:08:28.705 INFO:teuthology.orchestra.run.smithi178.stdout:1 2023-06-28T20:08:28.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:28.709 DEBUG:teuthology.orchestra.run.smithi178:> route 2023-06-28T20:08:28.768 INFO:teuthology.orchestra.run.smithi178.stdout:Kernel IP routing table 2023-06-28T20:08:28.768 INFO:teuthology.orchestra.run.smithi178.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-28T20:08:28.769 INFO:teuthology.orchestra.run.smithi178.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-06-28T20:08:28.769 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-06-28T20:08:28.772 INFO:teuthology.orchestra.run.smithi178.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-28T20:08:28.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:28.774 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec bash -c ' 2023-06-28T20:08:28.774 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2023-06-28T20:08:28.775 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-06-28T20:08:28.776 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-06-28T20:08:28.776 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-06-28T20:08:28.777 DEBUG:teuthology.orchestra.run.smithi178:> ') 2023-06-28T20:08:28.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:28.878 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2023-06-28T20:08:28.895 INFO:teuthology.orchestra.run.smithi178.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-06-28T20:08:28.895 INFO:teuthology.orchestra.run.smithi178.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-06-28T20:08:28.895 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-06-28T20:08:28.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:28.896 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set brx.0 up 2023-06-28T20:08:28.962 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-28T20:08:28.962 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-06-28T20:08:29.018 INFO:teuthology.orchestra.run.smithi178.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-06-28T20:08:29.018 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-28T20:08:29.019 DEBUG:teuthology.orchestra.run.smithi178:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-06-28T20:08:29.074 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-06-28T20:08:29.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:29.076 DEBUG:teuthology.orchestra.run.smithi178:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-06-28T20:08:29.159 INFO:teuthology.orchestra.run.smithi178.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-06-28T20:08:29.160 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-06-28T20:08:29.160 INFO:teuthology.orchestra.run.smithi178.stdout:invalid new device string format 2023-06-28T20:08:29.168 INFO:teuthology.orchestra.run.smithi178.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-06-28T20:08:29.169 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-06-28T20:08:29.170 INFO:teuthology.orchestra.run.smithi178.stdout:invalid new device string format 2023-06-28T20:08:29.170 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.178:3300,172.21.15.178:3301" 2023-06-28T20:08:29.171 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: switching to using v1 address with old syntax 2023-06-28T20:08:29.235 INFO:teuthology.orchestra.run.smithi178.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-06-28T20:08:29.235 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-06-28T20:08:29.236 INFO:teuthology.orchestra.run.smithi178.stdout:invalid new device string format 2023-06-28T20:08:29.236 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: resolved to: "172.21.15.43:3300,172.21.15.178:3300,172.21.15.178:3301" 2023-06-28T20:08:29.236 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: switching to using v1 address with old syntax 2023-06-28T20:08:29.236 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: resolved to: "172.21.15.43:6789,172.21.15.178:6789,172.21.15.178:6790" 2023-06-28T20:08:29.237 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.43:6789,172.21.15.178:6789,172.21.15.178:6790:/ 2023-06-28T20:08:29.237 INFO:teuthology.orchestra.run.smithi178.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=b8fd740d-00b9-4308-92a8-48f5fffe8078" will pass to kernel 2023-06-28T20:08:29.238 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-06-28T20:08:29.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:29.240 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-06-28T20:08:29.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:29.267 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-06-28T20:08:29.338 INFO:teuthology.orchestra.run.smithi178.stdout:client5164 2023-06-28T20:08:29.338 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:08:29.339 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:08:29.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:29.395 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-06-28T20:08:29.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:29.421 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 0 ./subdir) 2023-06-28T20:08:29.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:29.491 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.quota.max_bytes ./subdir) 2023-06-28T20:08:29.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-28T20:08:29.551 INFO:teuthology.orchestra.run.smithi043.stderr:./subdir: ceph.quota.max_bytes: No such attribute 2023-06-28T20:08:29.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:29.552 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 1K ./subdir) 2023-06-28T20:08:29.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-28T20:08:29.612 INFO:teuthology.orchestra.run.smithi043.stderr:setfattr: ./subdir: Invalid argument 2023-06-28T20:08:29.612 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-06-28T20:08:29.613 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-06-28T20:08:29.705 DEBUG:teuthology.orchestra.run.smithi178:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-06-28T20:08:29.742 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-06-28T20:08:29.743 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:08:29.743 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:08:29.763 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:08:29.764 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:08:29.822 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:08:29.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.989+0000 7f297ba57700 1 Processor -- start 2023-06-28T20:08:29.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.990+0000 7f297ba57700 1 -- start start 2023-06-28T20:08:29.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.990+0000 7f297ba57700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f29740b2170 0x7f29740d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:29.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.990+0000 7f297ba57700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f29740cf120 0x7f29740b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:29.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.990+0000 7f297ba57700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2974160270 con 0x7f29740d2b00 2023-06-28T20:08:29.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.990+0000 7f297ba57700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29740d0910 con 0x7f29740cf120 2023-06-28T20:08:29.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.990+0000 7f297ba57700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29740cf870 con 0x7f29740b2170 2023-06-28T20:08:29.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.991+0000 7f2978ff2700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f29740cf120 0x7f29740b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:29.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.991+0000 7f2978ff2700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f29740cf120 0x7f29740b5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55176/0 (socket says 172.21.15.43:55176) 2023-06-28T20:08:29.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.991+0000 7f2978ff2700 1 -- 172.21.15.43:0/2565296519 learned_addr learned my addr 172.21.15.43:0/2565296519 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:29.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.991+0000 7f29797f3700 1 --2- 172.21.15.43:0/2565296519 >> v2:172.21.15.178:3300/0 conn(0x7f29740b2170 0x7f29740d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:29.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.991+0000 7f2978ff2700 1 -- 172.21.15.43:0/2565296519 >> v2:172.21.15.178:3300/0 conn(0x7f29740b2170 msgr2=0x7f29740d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:29.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.991+0000 7f2978ff2700 1 --2- 172.21.15.43:0/2565296519 >> v2:172.21.15.178:3300/0 conn(0x7f29740b2170 0x7f29740d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:29.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.991+0000 7f2978ff2700 1 -- 172.21.15.43:0/2565296519 >> v1:172.21.15.178:6789/0 conn(0x7f29740d2b00 legacy=0x7f297406a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:29.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.991+0000 7f2978ff2700 1 -- 172.21.15.43:0/2565296519 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29740cf590 con 0x7f29740cf120 2023-06-28T20:08:29.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.991+0000 7f2978ff2700 1 --2- 172.21.15.43:0/2565296519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29740cf120 0x7f29740b5e30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f297408c2d0 tx=0x7f2968009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958a211ce8e1ce51 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:29.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.991+0000 7f2967fff700 1 -- 172.21.15.43:0/2565296519 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296800b860 con 0x7f29740cf120 2023-06-28T20:08:29.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.991+0000 7f2967fff700 1 -- 172.21.15.43:0/2565296519 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296800b9c0 con 0x7f29740cf120 2023-06-28T20:08:29.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.991+0000 7f2967fff700 1 -- 172.21.15.43:0/2565296519 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296800bd50 con 0x7f29740cf120 2023-06-28T20:08:30.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.992+0000 7f297ba57700 1 -- 172.21.15.43:0/2565296519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29740cf120 msgr2=0x7f29740b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:30.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.992+0000 7f297ba57700 1 --2- 172.21.15.43:0/2565296519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29740cf120 0x7f29740b5e30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f297408c2d0 tx=0x7f2968009c00 comp rx=0 tx=0).stop 2023-06-28T20:08:30.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.992+0000 7f297ba57700 1 -- 172.21.15.43:0/2565296519 shutdown_connections 2023-06-28T20:08:30.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.992+0000 7f297ba57700 1 --2- 172.21.15.43:0/2565296519 >> v2:172.21.15.178:3300/0 conn(0x7f29740b2170 0x7f29740d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.992+0000 7f297ba57700 1 --2- 172.21.15.43:0/2565296519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29740cf120 0x7f29740b5e30 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.992+0000 7f297ba57700 1 -- 172.21.15.43:0/2565296519 >> 172.21.15.43:0/2565296519 conn(0x7f297406c1e0 msgr2=0x7f297406c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:30.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.992+0000 7f297ba57700 1 -- 172.21.15.43:0/2565296519 shutdown_connections 2023-06-28T20:08:30.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.992+0000 7f297ba57700 1 -- 172.21.15.43:0/2565296519 wait complete. 2023-06-28T20:08:30.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.993+0000 7f297ba57700 1 Processor -- start 2023-06-28T20:08:30.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.993+0000 7f297ba57700 1 -- start start 2023-06-28T20:08:30.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.994+0000 7f297ba57700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29740b2170 0x7f29740c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:30.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.994+0000 7f297ba57700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29740cf120 0x7f2974150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:30.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.994+0000 7f297ba57700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29741510b0 0x7f2974159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:30.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.994+0000 7f297ba57700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29740c99d0 con 0x7f29740cf120 2023-06-28T20:08:30.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.994+0000 7f297ba57700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29740c96f0 con 0x7f29741510b0 2023-06-28T20:08:30.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.994+0000 7f297ba57700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29740c6cb0 con 0x7f29740b2170 2023-06-28T20:08:30.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.994+0000 7f2978ff2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29740cf120 0x7f2974150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:30.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.994+0000 7f2978ff2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29740cf120 0x7f2974150b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55184/0 (socket says 172.21.15.43:55184) 2023-06-28T20:08:30.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.994+0000 7f2978ff2700 1 -- 172.21.15.43:0/1546079993 learned_addr learned my addr 172.21.15.43:0/1546079993 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:30.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.994+0000 7f2979ff4700 1 --2- 172.21.15.43:0/1546079993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29741510b0 0x7f2974159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:30.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.994+0000 7f29797f3700 1 --2- 172.21.15.43:0/1546079993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29740b2170 0x7f29740c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:30.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.995+0000 7f2978ff2700 1 -- 172.21.15.43:0/1546079993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29740b2170 msgr2=0x7f29740c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:30.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.995+0000 7f2978ff2700 1 --2- 172.21.15.43:0/1546079993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29740b2170 0x7f29740c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.995+0000 7f2978ff2700 1 -- 172.21.15.43:0/1546079993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29741510b0 msgr2=0x7f2974159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:30.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.995+0000 7f2978ff2700 1 --2- 172.21.15.43:0/1546079993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29741510b0 0x7f2974159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.995+0000 7f2978ff2700 1 -- 172.21.15.43:0/1546079993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2968009550 con 0x7f29740cf120 2023-06-28T20:08:30.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.995+0000 7f2978ff2700 1 --2- 172.21.15.43:0/1546079993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29740cf120 0x7f2974150b70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2968009c40 tx=0x7f2968004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:30.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.995+0000 7f2965ffb700 1 -- 172.21.15.43:0/1546079993 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2968004300 con 0x7f29740cf120 2023-06-28T20:08:30.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.995+0000 7f2965ffb700 1 -- 172.21.15.43:0/1546079993 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296801f070 con 0x7f29740cf120 2023-06-28T20:08:30.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.995+0000 7f297ba57700 1 -- 172.21.15.43:0/1546079993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29740c6f90 con 0x7f29740cf120 2023-06-28T20:08:30.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.995+0000 7f2965ffb700 1 -- 172.21.15.43:0/1546079993 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29680036a0 con 0x7f29740cf120 2023-06-28T20:08:30.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.995+0000 7f297ba57700 1 -- 172.21.15.43:0/1546079993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29740c9330 con 0x7f29740cf120 2023-06-28T20:08:30.006 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.996+0000 7f297ba57700 1 -- 172.21.15.43:0/1546079993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29740c94c0 con 0x7f29740cf120 2023-06-28T20:08:30.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.997+0000 7f2965ffb700 1 -- 172.21.15.43:0/1546079993 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2968021030 con 0x7f29740cf120 2023-06-28T20:08:30.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.997+0000 7f2965ffb700 1 --2- 172.21.15.43:0/1546079993 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f295806ace0 0x7f295806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:30.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.997+0000 7f2965ffb700 1 -- 172.21.15.43:0/1546079993 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f296808a800 con 0x7f29740cf120 2023-06-28T20:08:30.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.997+0000 7f29797f3700 1 --2- 172.21.15.43:0/1546079993 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f295806ace0 0x7f295806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:30.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:29.997+0000 7f29797f3700 1 --2- 172.21.15.43:0/1546079993 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f295806ace0 0x7f295806d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f297415a140 tx=0x7f2970008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:30.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.002+0000 7f2965ffb700 1 -- 172.21.15.43:0/1546079993 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2968055a60 con 0x7f29740cf120 2023-06-28T20:08:30.205 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.201+0000 7f297ba57700 1 -- 172.21.15.43:0/1546079993 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29740c6b10 con 0x7f29740cf120 2023-06-28T20:08:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.203+0000 7f2965ffb700 1 -- 172.21.15.43:0/1546079993 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+18293 (secure 0 0 0) 0x7f29740c6b10 con 0x7f29740cf120 2023-06-28T20:08:30.207 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:30.208 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":35,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:08:21.654904+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:08:17.811454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-06-28T20:08:19.046351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:08:30.209 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.205+0000 7f297ba57700 1 -- 172.21.15.43:0/1546079993 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f295806ace0 msgr2=0x7f295806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:30.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.205+0000 7f297ba57700 1 --2- 172.21.15.43:0/1546079993 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f295806ace0 0x7f295806d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f297415a140 tx=0x7f2970008040 comp rx=0 tx=0).stop 2023-06-28T20:08:30.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.205+0000 7f297ba57700 1 -- 172.21.15.43:0/1546079993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29740cf120 msgr2=0x7f2974150b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:30.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.205+0000 7f297ba57700 1 --2- 172.21.15.43:0/1546079993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29740cf120 0x7f2974150b70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2968009c40 tx=0x7f2968004750 comp rx=0 tx=0).stop 2023-06-28T20:08:30.210 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.205+0000 7f297ba57700 1 -- 172.21.15.43:0/1546079993 shutdown_connections 2023-06-28T20:08:30.211 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.205+0000 7f297ba57700 1 --2- 172.21.15.43:0/1546079993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29741510b0 0x7f2974159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.205+0000 7f297ba57700 1 --2- 172.21.15.43:0/1546079993 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f295806ace0 0x7f295806d190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.205+0000 7f297ba57700 1 --2- 172.21.15.43:0/1546079993 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29740b2170 0x7f29740c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.205+0000 7f297ba57700 1 --2- 172.21.15.43:0/1546079993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29740cf120 0x7f2974150b70 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.205+0000 7f297ba57700 1 -- 172.21.15.43:0/1546079993 >> 172.21.15.43:0/1546079993 conn(0x7f297406c1e0 msgr2=0x7f297406c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:30.212 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.205+0000 7f297ba57700 1 -- 172.21.15.43:0/1546079993 shutdown_connections 2023-06-28T20:08:30.213 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.205+0000 7f297ba57700 1 -- 172.21.15.43:0/1546079993 wait complete. 2023-06-28T20:08:30.222 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-28T20:07:13.580969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-28T20:07:14.038358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-28T20:08:17.811454+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-06-28T20:08:19.046351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-28T20:08:30.224 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-28T20:08:30.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.348+0000 7f3d25eda700 1 Processor -- start 2023-06-28T20:08:30.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.349+0000 7f3d25eda700 1 -- start start 2023-06-28T20:08:30.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.349+0000 7f3d25eda700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3d2006d310 0x7f3d2006cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:30.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.349+0000 7f3d25eda700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d2015cae0 0x7f3d2006d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:30.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.349+0000 7f3d25eda700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d200d0680 con 0x7f3d200cfe00 2023-06-28T20:08:30.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.349+0000 7f3d25eda700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d2006dc20 con 0x7f3d2006d310 2023-06-28T20:08:30.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.349+0000 7f3d25eda700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2006bb20 con 0x7f3d2015cae0 2023-06-28T20:08:30.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.350+0000 7f3d1effd700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d2015cae0 0x7f3d2006d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:30.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.350+0000 7f3d1f7fe700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3d2006d310 0x7f3d2006cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:30.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.350+0000 7f3d1f7fe700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f3d2006d310 0x7f3d2006cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:41254/0 (socket says 172.21.15.43:41254) 2023-06-28T20:08:30.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.350+0000 7f3d1ffff700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f3d200cfe00 0x7f3d200ad7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:43504/0 (socket says 172.21.15.43:43504) 2023-06-28T20:08:30.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.350+0000 7f3d1effd700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d2015cae0 0x7f3d2006d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.43:39164/0 (socket says 172.21.15.43:39164) 2023-06-28T20:08:30.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.350+0000 7f3d1f7fe700 1 -- 172.21.15.43:0/3861706601 learned_addr learned my addr 172.21.15.43:0/3861706601 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:30.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.350+0000 7f3d1e7fc700 1 -- 172.21.15.43:0/3861706601 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486394406 0 0) 0x7f3d200d0680 con 0x7f3d200cfe00 2023-06-28T20:08:30.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.350+0000 7f3d1e7fc700 1 -- 172.21.15.43:0/3861706601 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d0c0035b0 con 0x7f3d200cfe00 2023-06-28T20:08:30.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.350+0000 7f3d1f7fe700 1 -- 172.21.15.43:0/3861706601 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d2015cae0 msgr2=0x7f3d2006d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:30.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.350+0000 7f3d1f7fe700 1 --2- 172.21.15.43:0/3861706601 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d2015cae0 0x7f3d2006d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.350+0000 7f3d1f7fe700 1 -- 172.21.15.43:0/3861706601 >> v1:172.21.15.178:6789/0 conn(0x7f3d200cfe00 legacy=0x7f3d200ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:30.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.350+0000 7f3d1f7fe700 1 -- 172.21.15.43:0/3861706601 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2006be00 con 0x7f3d2006d310 2023-06-28T20:08:30.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.350+0000 7f3d1effd700 1 --2- 172.21.15.43:0/3861706601 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d2015cae0 0x7f3d2006d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:08:30.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.350+0000 7f3d1f7fe700 1 --2- 172.21.15.43:0/3861706601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d2006d310 0x7f3d2006cbc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3d10006fd0 tx=0x7f3d10009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e08ef2b00f9ecf4 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:30.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.351+0000 7f3d1e7fc700 1 -- 172.21.15.43:0/3861706601 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d10005e30 con 0x7f3d2006d310 2023-06-28T20:08:30.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.351+0000 7f3d1e7fc700 1 -- 172.21.15.43:0/3861706601 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d10004030 con 0x7f3d2006d310 2023-06-28T20:08:30.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.351+0000 7f3d1e7fc700 1 -- 172.21.15.43:0/3861706601 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d100043e0 con 0x7f3d2006d310 2023-06-28T20:08:30.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.351+0000 7f3d25eda700 1 -- 172.21.15.43:0/3861706601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d2006d310 msgr2=0x7f3d2006cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:30.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.351+0000 7f3d25eda700 1 --2- 172.21.15.43:0/3861706601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d2006d310 0x7f3d2006cbc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3d10006fd0 tx=0x7f3d10009bd0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.351+0000 7f3d25eda700 1 -- 172.21.15.43:0/3861706601 shutdown_connections 2023-06-28T20:08:30.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.351+0000 7f3d25eda700 1 --2- 172.21.15.43:0/3861706601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d2006d310 0x7f3d2006cbc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.351+0000 7f3d25eda700 1 --2- 172.21.15.43:0/3861706601 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d2015cae0 0x7f3d2006d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.351+0000 7f3d25eda700 1 -- 172.21.15.43:0/3861706601 >> 172.21.15.43:0/3861706601 conn(0x7f3d200664b0 msgr2=0x7f3d200d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:30.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.351+0000 7f3d25eda700 1 -- 172.21.15.43:0/3861706601 shutdown_connections 2023-06-28T20:08:30.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.351+0000 7f3d25eda700 1 -- 172.21.15.43:0/3861706601 wait complete. 2023-06-28T20:08:30.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.352+0000 7f3d25eda700 1 Processor -- start 2023-06-28T20:08:30.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.353+0000 7f3d25eda700 1 -- start start 2023-06-28T20:08:30.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.353+0000 7f3d25eda700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d2006d310 0x7f3d2015dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:30.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.353+0000 7f3d25eda700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d200cfe00 0x7f3d20157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:30.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.353+0000 7f3d25eda700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d2015cae0 0x7f3d2015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:30.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.353+0000 7f3d25eda700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2015fdb0 con 0x7f3d2006d310 2023-06-28T20:08:30.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.353+0000 7f3d25eda700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d201f31c0 con 0x7f3d200cfe00 2023-06-28T20:08:30.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.353+0000 7f3d25eda700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d201f34a0 con 0x7f3d2015cae0 2023-06-28T20:08:30.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.353+0000 7f3d1f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d2006d310 0x7f3d2015dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:30.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.353+0000 7f3d1f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d2006d310 0x7f3d2015dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55196/0 (socket says 172.21.15.43:55196) 2023-06-28T20:08:30.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.353+0000 7f3d1f7fe700 1 -- 172.21.15.43:0/439108137 learned_addr learned my addr 172.21.15.43:0/439108137 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:30.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.353+0000 7f3d1ffff700 1 --2- 172.21.15.43:0/439108137 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d2015cae0 0x7f3d2015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:30.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.354+0000 7f3d1effd700 1 --2- 172.21.15.43:0/439108137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d200cfe00 0x7f3d20157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:30.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.354+0000 7f3d1f7fe700 1 -- 172.21.15.43:0/439108137 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d2015cae0 msgr2=0x7f3d2015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:30.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.354+0000 7f3d1f7fe700 1 --2- 172.21.15.43:0/439108137 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d2015cae0 0x7f3d2015f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.354+0000 7f3d1f7fe700 1 -- 172.21.15.43:0/439108137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d200cfe00 msgr2=0x7f3d20157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:30.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.354+0000 7f3d1f7fe700 1 --2- 172.21.15.43:0/439108137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d200cfe00 0x7f3d20157dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.354+0000 7f3d1f7fe700 1 -- 172.21.15.43:0/439108137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d080094a0 con 0x7f3d2006d310 2023-06-28T20:08:30.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.354+0000 7f3d1ffff700 1 --2- 172.21.15.43:0/439108137 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d2015cae0 0x7f3d2015f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:30.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.354+0000 7f3d1f7fe700 1 --2- 172.21.15.43:0/439108137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d2006d310 0x7f3d2015dbf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009c10 tx=0x7f3d10004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:30.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.354+0000 7f3d17fff700 1 -- 172.21.15.43:0/439108137 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d10003c70 con 0x7f3d2006d310 2023-06-28T20:08:30.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.354+0000 7f3d17fff700 1 -- 172.21.15.43:0/439108137 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d10003dd0 con 0x7f3d2006d310 2023-06-28T20:08:30.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.354+0000 7f3d25eda700 1 -- 172.21.15.43:0/439108137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d10009500 con 0x7f3d2006d310 2023-06-28T20:08:30.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.354+0000 7f3d17fff700 1 -- 172.21.15.43:0/439108137 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1001c640 con 0x7f3d2006d310 2023-06-28T20:08:30.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.354+0000 7f3d25eda700 1 -- 172.21.15.43:0/439108137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d201f3a60 con 0x7f3d2006d310 2023-06-28T20:08:30.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.355+0000 7f3d25eda700 1 -- 172.21.15.43:0/439108137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d201f3c50 con 0x7f3d2006d310 2023-06-28T20:08:30.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.356+0000 7f3d17fff700 1 -- 172.21.15.43:0/439108137 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3d1001c7a0 con 0x7f3d2006d310 2023-06-28T20:08:30.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.356+0000 7f3d17fff700 1 --2- 172.21.15.43:0/439108137 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f3d0006ace0 0x7f3d0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:30.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.356+0000 7f3d17fff700 1 -- 172.21.15.43:0/439108137 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3d1008a3f0 con 0x7f3d2006d310 2023-06-28T20:08:30.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.356+0000 7f3d1effd700 1 --2- 172.21.15.43:0/439108137 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f3d0006ace0 0x7f3d0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:30.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.357+0000 7f3d1effd700 1 --2- 172.21.15.43:0/439108137 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f3d0006ace0 0x7f3d0006d190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3d201e64a0 tx=0x7f3d08009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:30.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.361+0000 7f3d17fff700 1 -- 172.21.15.43:0/439108137 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3d10055530 con 0x7f3d2006d310 2023-06-28T20:08:30.561 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.557+0000 7f3d25eda700 1 -- 172.21.15.43:0/439108137 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3d200728b0 con 0x7f3d2006d310 2023-06-28T20:08:30.562 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.558+0000 7f3d17fff700 1 -- 172.21.15.43:0/439108137 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v35) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3d10058b50 con 0x7f3d2006d310 2023-06-28T20:08:30.562 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-06-28T20:08:30.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.560+0000 7f3d25eda700 1 -- 172.21.15.43:0/439108137 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f3d0006ace0 msgr2=0x7f3d0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:30.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.560+0000 7f3d25eda700 1 --2- 172.21.15.43:0/439108137 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f3d0006ace0 0x7f3d0006d190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3d201e64a0 tx=0x7f3d08009210 comp rx=0 tx=0).stop 2023-06-28T20:08:30.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.560+0000 7f3d25eda700 1 -- 172.21.15.43:0/439108137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d2006d310 msgr2=0x7f3d2015dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:30.564 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.560+0000 7f3d25eda700 1 --2- 172.21.15.43:0/439108137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d2006d310 0x7f3d2015dbf0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009c10 tx=0x7f3d10004750 comp rx=0 tx=0).stop 2023-06-28T20:08:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.560+0000 7f3d25eda700 1 -- 172.21.15.43:0/439108137 shutdown_connections 2023-06-28T20:08:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.560+0000 7f3d25eda700 1 --2- 172.21.15.43:0/439108137 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f3d0006ace0 0x7f3d0006d190 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.560+0000 7f3d25eda700 1 --2- 172.21.15.43:0/439108137 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3d2015cae0 0x7f3d2015f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.560+0000 7f3d25eda700 1 --2- 172.21.15.43:0/439108137 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d2006d310 0x7f3d2015dbf0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.560+0000 7f3d25eda700 1 --2- 172.21.15.43:0/439108137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d200cfe00 0x7f3d20157dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.560+0000 7f3d25eda700 1 -- 172.21.15.43:0/439108137 >> 172.21.15.43:0/439108137 conn(0x7f3d200664b0 msgr2=0x7f3d200af3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.560+0000 7f3d25eda700 1 -- 172.21.15.43:0/439108137 shutdown_connections 2023-06-28T20:08:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.560+0000 7f3d25eda700 1 -- 172.21.15.43:0/439108137 wait complete. 2023-06-28T20:08:30.576 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-28T20:08:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.700+0000 7fa86bb36700 1 Processor -- start 2023-06-28T20:08:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.700+0000 7fa86bb36700 1 -- start start 2023-06-28T20:08:30.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.701+0000 7fa86bb36700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8640caff0 0x7fa86406b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:30.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.701+0000 7fa86bb36700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa8640cb3c0 0x7fa86406c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:30.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.701+0000 7fa86bb36700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa86406a020 con 0x7fa86406c5e0 2023-06-28T20:08:30.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.701+0000 7fa86bb36700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa86406ca60 con 0x7fa8640cb3c0 2023-06-28T20:08:30.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.701+0000 7fa86bb36700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86406a300 con 0x7fa8640caff0 2023-06-28T20:08:30.706 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.701+0000 7fa8690d1700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa8640cb3c0 0x7fa86406c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:30.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.701+0000 7fa8690d1700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa8640cb3c0 0x7fa86406c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55204/0 (socket says 172.21.15.43:55204) 2023-06-28T20:08:30.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.701+0000 7fa8690d1700 1 -- 172.21.15.43:0/304239268 learned_addr learned my addr 172.21.15.43:0/304239268 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:30.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.701+0000 7fa8698d2700 1 --2- 172.21.15.43:0/304239268 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8640caff0 0x7fa86406b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:30.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.701+0000 7fa8690d1700 1 -- 172.21.15.43:0/304239268 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8640caff0 msgr2=0x7fa86406b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:30.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.701+0000 7fa8690d1700 1 --2- 172.21.15.43:0/304239268 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8640caff0 0x7fa86406b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.701+0000 7fa8690d1700 1 -- 172.21.15.43:0/304239268 >> v1:172.21.15.178:6789/0 conn(0x7fa86406c5e0 legacy=0x7fa86415df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:30.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.702+0000 7fa8690d1700 1 -- 172.21.15.43:0/304239268 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86406a5e0 con 0x7fa8640cb3c0 2023-06-28T20:08:30.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.702+0000 7fa8690d1700 1 --2- 172.21.15.43:0/304239268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8640cb3c0 0x7fa86406c0a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa8600098f0 tx=0x7fa860009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa3b3d94a5ca841 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:30.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.702+0000 7fa8688d0700 1 -- 172.21.15.43:0/304239268 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860011830 con 0x7fa8640cb3c0 2023-06-28T20:08:30.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.702+0000 7fa8688d0700 1 -- 172.21.15.43:0/304239268 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa860011990 con 0x7fa8640cb3c0 2023-06-28T20:08:30.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.702+0000 7fa8688d0700 1 -- 172.21.15.43:0/304239268 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860011d20 con 0x7fa8640cb3c0 2023-06-28T20:08:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.702+0000 7fa86bb36700 1 -- 172.21.15.43:0/304239268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8640cb3c0 msgr2=0x7fa86406c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.702+0000 7fa86bb36700 1 --2- 172.21.15.43:0/304239268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8640cb3c0 0x7fa86406c0a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa8600098f0 tx=0x7fa860009c00 comp rx=0 tx=0).stop 2023-06-28T20:08:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.702+0000 7fa86bb36700 1 -- 172.21.15.43:0/304239268 shutdown_connections 2023-06-28T20:08:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.702+0000 7fa86bb36700 1 --2- 172.21.15.43:0/304239268 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8640caff0 0x7fa86406b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.702+0000 7fa86bb36700 1 --2- 172.21.15.43:0/304239268 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8640cb3c0 0x7fa86406c0a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.702+0000 7fa86bb36700 1 -- 172.21.15.43:0/304239268 >> 172.21.15.43:0/304239268 conn(0x7fa8640b1a50 msgr2=0x7fa8640cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.703+0000 7fa86bb36700 1 -- 172.21.15.43:0/304239268 shutdown_connections 2023-06-28T20:08:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.703+0000 7fa86bb36700 1 -- 172.21.15.43:0/304239268 wait complete. 2023-06-28T20:08:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.704+0000 7fa86bb36700 1 Processor -- start 2023-06-28T20:08:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.704+0000 7fa86bb36700 1 -- start start 2023-06-28T20:08:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.704+0000 7fa86bb36700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa86406c5e0 0x7fa8641570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.704+0000 7fa86bb36700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8640caff0 0x7fa864156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.705+0000 7fa86bb36700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8640cb3c0 0x7fa8640b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.705+0000 7fa86bb36700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8640b20e0 con 0x7fa86406c5e0 2023-06-28T20:08:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.705+0000 7fa86bb36700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8641e1bf0 con 0x7fa8640caff0 2023-06-28T20:08:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.705+0000 7fa86bb36700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8640b2380 con 0x7fa8640cb3c0 2023-06-28T20:08:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.705+0000 7fa8698d2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa86406c5e0 0x7fa8641570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.705+0000 7fa8698d2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa86406c5e0 0x7fa8641570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55208/0 (socket says 172.21.15.43:55208) 2023-06-28T20:08:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.705+0000 7fa8698d2700 1 -- 172.21.15.43:0/641350028 learned_addr learned my addr 172.21.15.43:0/641350028 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.705+0000 7fa8690d1700 1 --2- 172.21.15.43:0/641350028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8640caff0 0x7fa864156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.705+0000 7fa86a0d3700 1 --2- 172.21.15.43:0/641350028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8640cb3c0 0x7fa8640b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.705+0000 7fa8698d2700 1 -- 172.21.15.43:0/641350028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8640cb3c0 msgr2=0x7fa8640b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.705+0000 7fa8698d2700 1 --2- 172.21.15.43:0/641350028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8640cb3c0 0x7fa8640b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.705+0000 7fa8698d2700 1 -- 172.21.15.43:0/641350028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8640caff0 msgr2=0x7fa864156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.705+0000 7fa8698d2700 1 --2- 172.21.15.43:0/641350028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8640caff0 0x7fa864156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.705+0000 7fa8698d2700 1 -- 172.21.15.43:0/641350028 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa860009550 con 0x7fa86406c5e0 2023-06-28T20:08:30.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.705+0000 7fa86a0d3700 1 --2- 172.21.15.43:0/641350028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8640cb3c0 0x7fa8640b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:30.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.705+0000 7fa8698d2700 1 --2- 172.21.15.43:0/641350028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa86406c5e0 0x7fa8641570e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa85400e9b0 tx=0x7fa85400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:30.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.705+0000 7fa85a7fc700 1 -- 172.21.15.43:0/641350028 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854009710 con 0x7fa86406c5e0 2023-06-28T20:08:30.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.705+0000 7fa85a7fc700 1 -- 172.21.15.43:0/641350028 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85400c850 con 0x7fa86406c5e0 2023-06-28T20:08:30.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.705+0000 7fa86bb36700 1 -- 172.21.15.43:0/641350028 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8640b2660 con 0x7fa86406c5e0 2023-06-28T20:08:30.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.705+0000 7fa85a7fc700 1 -- 172.21.15.43:0/641350028 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85400cbc0 con 0x7fa86406c5e0 2023-06-28T20:08:30.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.705+0000 7fa86bb36700 1 -- 172.21.15.43:0/641350028 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa864156d70 con 0x7fa86406c5e0 2023-06-28T20:08:30.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.706+0000 7fa86bb36700 1 -- 172.21.15.43:0/641350028 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8640b2cb0 con 0x7fa86406c5e0 2023-06-28T20:08:30.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.707+0000 7fa85a7fc700 1 -- 172.21.15.43:0/641350028 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa85400cd20 con 0x7fa86406c5e0 2023-06-28T20:08:30.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.707+0000 7fa85a7fc700 1 --2- 172.21.15.43:0/641350028 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa84806ace0 0x7fa84806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:30.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.707+0000 7fa85a7fc700 1 -- 172.21.15.43:0/641350028 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa854089510 con 0x7fa86406c5e0 2023-06-28T20:08:30.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.707+0000 7fa8690d1700 1 --2- 172.21.15.43:0/641350028 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa84806ace0 0x7fa84806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:30.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.708+0000 7fa8690d1700 1 --2- 172.21.15.43:0/641350028 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa84806ace0 0x7fa84806d190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa860009c40 tx=0x7fa86001d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:30.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.712+0000 7fa85a7fc700 1 -- 172.21.15.43:0/641350028 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa8540545c0 con 0x7fa86406c5e0 2023-06-28T20:08:30.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.908+0000 7fa86bb36700 1 -- 172.21.15.43:0/641350028 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa8640728b0 con 0x7fa86406c5e0 2023-06-28T20:08:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.909+0000 7fa85a7fc700 1 -- 172.21.15.43:0/641350028 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa854057be0 con 0x7fa86406c5e0 2023-06-28T20:08:30.913 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-06-28T20:08:30.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.910+0000 7fa86bb36700 1 -- 172.21.15.43:0/641350028 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa84806ace0 msgr2=0x7fa84806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:30.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.910+0000 7fa86bb36700 1 --2- 172.21.15.43:0/641350028 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa84806ace0 0x7fa84806d190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa860009c40 tx=0x7fa86001d040 comp rx=0 tx=0).stop 2023-06-28T20:08:30.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.910+0000 7fa86bb36700 1 -- 172.21.15.43:0/641350028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa86406c5e0 msgr2=0x7fa8641570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:30.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.910+0000 7fa86bb36700 1 --2- 172.21.15.43:0/641350028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa86406c5e0 0x7fa8641570e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa85400e9b0 tx=0x7fa85400ecc0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.910+0000 7fa86bb36700 1 -- 172.21.15.43:0/641350028 shutdown_connections 2023-06-28T20:08:30.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.910+0000 7fa86bb36700 1 --2- 172.21.15.43:0/641350028 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa84806ace0 0x7fa84806d190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.910+0000 7fa86bb36700 1 --2- 172.21.15.43:0/641350028 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa8640cb3c0 0x7fa8640b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.910+0000 7fa86bb36700 1 --2- 172.21.15.43:0/641350028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa86406c5e0 0x7fa8641570e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.910+0000 7fa86bb36700 1 --2- 172.21.15.43:0/641350028 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8640caff0 0x7fa864156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:30.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.910+0000 7fa86bb36700 1 -- 172.21.15.43:0/641350028 >> 172.21.15.43:0/641350028 conn(0x7fa8640b1a50 msgr2=0x7fa8640c6380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:30.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.911+0000 7fa86bb36700 1 -- 172.21.15.43:0/641350028 shutdown_connections 2023-06-28T20:08:30.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:30.911+0000 7fa86bb36700 1 -- 172.21.15.43:0/641350028 wait complete. 2023-06-28T20:08:30.928 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-28T20:08:31.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.052+0000 7f09c9917700 1 Processor -- start 2023-06-28T20:08:31.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.052+0000 7f09c9917700 1 -- start start 2023-06-28T20:08:31.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.053+0000 7f09c9917700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f09c40cf830 0x7f09c40cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:31.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.053+0000 7f09c9917700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c40b89f0 con 0x7f09c40d0a80 2023-06-28T20:08:31.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.053+0000 7f09c9917700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c40d0f00 con 0x7f09c415cf10 2023-06-28T20:08:31.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.053+0000 7f09c9917700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09c40d0140 con 0x7f09c40cf830 2023-06-28T20:08:31.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.053+0000 7f09c2ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f09c40cf830 0x7f09c40cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:31.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.053+0000 7f09c27fc700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f09c40d0a80 0x7f09c40d1c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44406/0 (socket says 172.21.15.43:44406) 2023-06-28T20:08:31.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.053+0000 7f09c2ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f09c40cf830 0x7f09c40cfc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55216/0 (socket says 172.21.15.43:55216) 2023-06-28T20:08:31.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.053+0000 7f09c27fc700 1 -- 172.21.15.43:0/2575872792 learned_addr learned my addr 172.21.15.43:0/2575872792 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:31.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.053+0000 7f09c2ffd700 1 -- 172.21.15.43:0/2575872792 >> v1:172.21.15.178:6789/0 conn(0x7f09c415cf10 legacy=0x7f09c40b82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:31.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.053+0000 7f09c2ffd700 1 -- 172.21.15.43:0/2575872792 >> v1:172.21.15.43:6789/0 conn(0x7f09c40d0a80 legacy=0x7f09c40d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:31.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.053+0000 7f09c2ffd700 1 -- 172.21.15.43:0/2575872792 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c40b8c80 con 0x7f09c40cf830 2023-06-28T20:08:31.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.053+0000 7f09c1ffb700 1 -- 172.21.15.43:0/2575872792 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1779071322 0 0) 0x7f09c40b89f0 con 0x7f09c40d0a80 2023-06-28T20:08:31.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.054+0000 7f09c2ffd700 1 --2- 172.21.15.43:0/2575872792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09c40cf830 0x7f09c40cfc00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009840 tx=0x7f09b4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19dc61d13fedec3 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:31.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.054+0000 7f09c1ffb700 1 -- 172.21.15.43:0/2575872792 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b4011830 con 0x7f09c40cf830 2023-06-28T20:08:31.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.054+0000 7f09c1ffb700 1 -- 172.21.15.43:0/2575872792 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b4011990 con 0x7f09c40cf830 2023-06-28T20:08:31.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.054+0000 7f09c1ffb700 1 -- 172.21.15.43:0/2575872792 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b4011d20 con 0x7f09c40cf830 2023-06-28T20:08:31.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.054+0000 7f09c9917700 1 -- 172.21.15.43:0/2575872792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09c40cf830 msgr2=0x7f09c40cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:31.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.054+0000 7f09c9917700 1 --2- 172.21.15.43:0/2575872792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09c40cf830 0x7f09c40cfc00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009840 tx=0x7f09b4009b50 comp rx=0 tx=0).stop 2023-06-28T20:08:31.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.054+0000 7f09c9917700 1 -- 172.21.15.43:0/2575872792 shutdown_connections 2023-06-28T20:08:31.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.054+0000 7f09c9917700 1 --2- 172.21.15.43:0/2575872792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09c40cf830 0x7f09c40cfc00 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.054+0000 7f09c9917700 1 -- 172.21.15.43:0/2575872792 >> 172.21.15.43:0/2575872792 conn(0x7f09c406a640 msgr2=0x7f09c406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:31.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.054+0000 7f09c9917700 1 -- 172.21.15.43:0/2575872792 shutdown_connections 2023-06-28T20:08:31.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.054+0000 7f09c9917700 1 -- 172.21.15.43:0/2575872792 wait complete. 2023-06-28T20:08:31.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.056+0000 7f09c9917700 1 Processor -- start 2023-06-28T20:08:31.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.056+0000 7f09c9917700 1 -- start start 2023-06-28T20:08:31.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.057+0000 7f09c9917700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09c40cf830 0x7f09c40cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:31.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.057+0000 7f09c9917700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09c40d0a80 0x7f09c40cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:31.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.057+0000 7f09c9917700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09c415cf10 0x7f09c414a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:31.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.057+0000 7f09c9917700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c40b5070 con 0x7f09c40d0a80 2023-06-28T20:08:31.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.057+0000 7f09c9917700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c414c2a0 con 0x7f09c40cf830 2023-06-28T20:08:31.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.057+0000 7f09c9917700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c41f5f40 con 0x7f09c415cf10 2023-06-28T20:08:31.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.057+0000 7f09c27fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09c40d0a80 0x7f09c40cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:31.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.057+0000 7f09c27fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09c40d0a80 0x7f09c40cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55224/0 (socket says 172.21.15.43:55224) 2023-06-28T20:08:31.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.057+0000 7f09c27fc700 1 -- 172.21.15.43:0/3432752569 learned_addr learned my addr 172.21.15.43:0/3432752569 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:31.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.057+0000 7f09c2ffd700 1 --2- 172.21.15.43:0/3432752569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09c40cf830 0x7f09c40cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:31.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.057+0000 7f09c37fe700 1 --2- 172.21.15.43:0/3432752569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09c415cf10 0x7f09c414a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:31.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.057+0000 7f09c27fc700 1 -- 172.21.15.43:0/3432752569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09c415cf10 msgr2=0x7f09c414a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:31.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.057+0000 7f09c27fc700 1 --2- 172.21.15.43:0/3432752569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09c415cf10 0x7f09c414a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.057+0000 7f09c27fc700 1 -- 172.21.15.43:0/3432752569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09c40cf830 msgr2=0x7f09c40cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:31.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.057+0000 7f09c27fc700 1 --2- 172.21.15.43:0/3432752569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09c40cf830 0x7f09c40cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.057+0000 7f09c27fc700 1 -- 172.21.15.43:0/3432752569 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b40094a0 con 0x7f09c40d0a80 2023-06-28T20:08:31.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.057+0000 7f09c27fc700 1 --2- 172.21.15.43:0/3432752569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09c40d0a80 0x7f09c40cd6c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f09b800c450 tx=0x7f09b800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:31.078 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.059+0000 7f09abfff700 1 -- 172.21.15.43:0/3432752569 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b8004c80 con 0x7f09c40d0a80 2023-06-28T20:08:31.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.059+0000 7f09c9917700 1 -- 172.21.15.43:0/3432752569 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c41f6220 con 0x7f09c40d0a80 2023-06-28T20:08:31.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.059+0000 7f09abfff700 1 -- 172.21.15.43:0/3432752569 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b8004e00 con 0x7f09c40d0a80 2023-06-28T20:08:31.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.059+0000 7f09abfff700 1 -- 172.21.15.43:0/3432752569 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b8007310 con 0x7f09c40d0a80 2023-06-28T20:08:31.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.059+0000 7f09c9917700 1 -- 172.21.15.43:0/3432752569 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c41f67e0 con 0x7f09c40d0a80 2023-06-28T20:08:31.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.060+0000 7f09c9917700 1 -- 172.21.15.43:0/3432752569 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c414c410 con 0x7f09c40d0a80 2023-06-28T20:08:31.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.060+0000 7f09abfff700 1 -- 172.21.15.43:0/3432752569 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f09b8007470 con 0x7f09c40d0a80 2023-06-28T20:08:31.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.061+0000 7f09abfff700 1 --2- 172.21.15.43:0/3432752569 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f09b006ace0 0x7f09b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:31.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.061+0000 7f09c2ffd700 1 --2- 172.21.15.43:0/3432752569 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f09b006ace0 0x7f09b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:31.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.061+0000 7f09abfff700 1 -- 172.21.15.43:0/3432752569 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f09b8089000 con 0x7f09c40d0a80 2023-06-28T20:08:31.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.061+0000 7f09c2ffd700 1 --2- 172.21.15.43:0/3432752569 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f09b006ace0 0x7f09b006d190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f09b40090b0 tx=0x7f09b401c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:31.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.066+0000 7f09abfff700 1 -- 172.21.15.43:0/3432752569 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f09b8054030 con 0x7f09c40d0a80 2023-06-28T20:08:31.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.265+0000 7f09c9917700 1 -- 172.21.15.43:0/3432752569 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f09c40728b0 con 0x7f09c40d0a80 2023-06-28T20:08:31.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.265+0000 7f09abfff700 1 -- 172.21.15.43:0/3432752569 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7f09b800ce60 con 0x7f09c40d0a80 2023-06-28T20:08:31.269 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-06-28T20:08:31.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.267+0000 7f09c9917700 1 -- 172.21.15.43:0/3432752569 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f09b006ace0 msgr2=0x7f09b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:31.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.267+0000 7f09c9917700 1 --2- 172.21.15.43:0/3432752569 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f09b006ace0 0x7f09b006d190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f09b40090b0 tx=0x7f09b401c040 comp rx=0 tx=0).stop 2023-06-28T20:08:31.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.267+0000 7f09c9917700 1 -- 172.21.15.43:0/3432752569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09c40d0a80 msgr2=0x7f09c40cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.267+0000 7f09c9917700 1 --2- 172.21.15.43:0/3432752569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09c40d0a80 0x7f09c40cd6c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f09b800c450 tx=0x7f09b800c760 comp rx=0 tx=0).stop 2023-06-28T20:08:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.267+0000 7f09c9917700 1 -- 172.21.15.43:0/3432752569 shutdown_connections 2023-06-28T20:08:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.267+0000 7f09c9917700 1 --2- 172.21.15.43:0/3432752569 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f09b006ace0 0x7f09b006d190 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.267+0000 7f09c9917700 1 --2- 172.21.15.43:0/3432752569 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f09c415cf10 0x7f09c414a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.267+0000 7f09c9917700 1 --2- 172.21.15.43:0/3432752569 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09c40cf830 0x7f09c40cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.267+0000 7f09c9917700 1 --2- 172.21.15.43:0/3432752569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09c40d0a80 0x7f09c40cd6c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.267+0000 7f09c9917700 1 -- 172.21.15.43:0/3432752569 >> 172.21.15.43:0/3432752569 conn(0x7f09c406a640 msgr2=0x7f09c415d430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.267+0000 7f09c9917700 1 -- 172.21.15.43:0/3432752569 shutdown_connections 2023-06-28T20:08:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.267+0000 7f09c9917700 1 -- 172.21.15.43:0/3432752569 wait complete. 2023-06-28T20:08:31.283 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-28T20:08:31.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.407+0000 7f1e768ca700 1 Processor -- start 2023-06-28T20:08:31.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.407+0000 7f1e768ca700 1 -- start start 2023-06-28T20:08:31.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.408+0000 7f1e768ca700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1e701592d0 0x7f1e700b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:31.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.408+0000 7f1e768ca700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1e70071900 0x7f1e700b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:31.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.408+0000 7f1e768ca700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e70070b40 con 0x7f1e701596a0 2023-06-28T20:08:31.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.408+0000 7f1e768ca700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e70070e20 con 0x7f1e70071900 2023-06-28T20:08:31.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.408+0000 7f1e768ca700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e70071100 con 0x7f1e701592d0 2023-06-28T20:08:31.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.408+0000 7f1e74e67700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1e70071900 0x7f1e700b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:31.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.408+0000 7f1e74e67700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1e70071900 0x7f1e700b1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55234/0 (socket says 172.21.15.43:55234) 2023-06-28T20:08:31.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.408+0000 7f1e74e67700 1 -- 172.21.15.43:0/29521553 learned_addr learned my addr 172.21.15.43:0/29521553 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:31.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.408+0000 7f1e6ffff700 1 --2- 172.21.15.43:0/29521553 >> v2:172.21.15.178:3300/0 conn(0x7f1e701592d0 0x7f1e700b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:31.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.409+0000 7f1e74e67700 1 -- 172.21.15.43:0/29521553 >> v2:172.21.15.178:3300/0 conn(0x7f1e701592d0 msgr2=0x7f1e700b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:31.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.409+0000 7f1e74e67700 1 --2- 172.21.15.43:0/29521553 >> v2:172.21.15.178:3300/0 conn(0x7f1e701592d0 0x7f1e700b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.409+0000 7f1e74e67700 1 -- 172.21.15.43:0/29521553 >> v1:172.21.15.178:6789/0 conn(0x7f1e701596a0 legacy=0x7f1e7015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:31.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.409+0000 7f1e74e67700 1 -- 172.21.15.43:0/29521553 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e700713e0 con 0x7f1e70071900 2023-06-28T20:08:31.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.409+0000 7f1e74e67700 1 --2- 172.21.15.43:0/29521553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e70071900 0x7f1e700b1f10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1e700d0bf0 tx=0x7f1e6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95941e97087a819f server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:31.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.409+0000 7f1e6effd700 1 -- 172.21.15.43:0/29521553 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6800e070 con 0x7f1e70071900 2023-06-28T20:08:31.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.409+0000 7f1e6effd700 1 -- 172.21.15.43:0/29521553 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6800b910 con 0x7f1e70071900 2023-06-28T20:08:31.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.409+0000 7f1e6effd700 1 -- 172.21.15.43:0/29521553 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6800bcc0 con 0x7f1e70071900 2023-06-28T20:08:31.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.410+0000 7f1e768ca700 1 -- 172.21.15.43:0/29521553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e70071900 msgr2=0x7f1e700b1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:31.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.410+0000 7f1e768ca700 1 --2- 172.21.15.43:0/29521553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e70071900 0x7f1e700b1f10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1e700d0bf0 tx=0x7f1e6800b330 comp rx=0 tx=0).stop 2023-06-28T20:08:31.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.410+0000 7f1e768ca700 1 -- 172.21.15.43:0/29521553 shutdown_connections 2023-06-28T20:08:31.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.410+0000 7f1e768ca700 1 --2- 172.21.15.43:0/29521553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e70071900 0x7f1e700b1f10 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.410+0000 7f1e768ca700 1 --2- 172.21.15.43:0/29521553 >> v2:172.21.15.178:3300/0 conn(0x7f1e701592d0 0x7f1e700b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.410+0000 7f1e768ca700 1 -- 172.21.15.43:0/29521553 >> 172.21.15.43:0/29521553 conn(0x7f1e7006b000 msgr2=0x7f1e7006b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:31.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.410+0000 7f1e768ca700 1 -- 172.21.15.43:0/29521553 shutdown_connections 2023-06-28T20:08:31.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.410+0000 7f1e768ca700 1 -- 172.21.15.43:0/29521553 wait complete. 2023-06-28T20:08:31.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.411+0000 7f1e768ca700 1 Processor -- start 2023-06-28T20:08:31.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.411+0000 7f1e768ca700 1 -- start start 2023-06-28T20:08:31.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.412+0000 7f1e768ca700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e70071900 0x7f1e7014bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:31.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.412+0000 7f1e768ca700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e701592d0 0x7f1e70158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:31.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.412+0000 7f1e768ca700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e701596a0 0x7f1e700b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:31.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.412+0000 7f1e768ca700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e70157d70 con 0x7f1e701592d0 2023-06-28T20:08:31.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.412+0000 7f1e768ca700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e70157160 con 0x7f1e701596a0 2023-06-28T20:08:31.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.412+0000 7f1e768ca700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e70157440 con 0x7f1e70071900 2023-06-28T20:08:31.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.412+0000 7f1e6f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e701592d0 0x7f1e70158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:31.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.412+0000 7f1e6f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e701592d0 0x7f1e70158200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55238/0 (socket says 172.21.15.43:55238) 2023-06-28T20:08:31.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.412+0000 7f1e6f7fe700 1 -- 172.21.15.43:0/2121259097 learned_addr learned my addr 172.21.15.43:0/2121259097 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:31.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.412+0000 7f1e6ffff700 1 --2- 172.21.15.43:0/2121259097 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e70071900 0x7f1e7014bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:31.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.412+0000 7f1e74e67700 1 --2- 172.21.15.43:0/2121259097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e701596a0 0x7f1e700b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:31.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.412+0000 7f1e6f7fe700 1 -- 172.21.15.43:0/2121259097 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e70071900 msgr2=0x7f1e7014bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:31.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.412+0000 7f1e6f7fe700 1 --2- 172.21.15.43:0/2121259097 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e70071900 0x7f1e7014bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.412+0000 7f1e6f7fe700 1 -- 172.21.15.43:0/2121259097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e701596a0 msgr2=0x7f1e700b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:31.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.412+0000 7f1e6f7fe700 1 --2- 172.21.15.43:0/2121259097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e701596a0 0x7f1e700b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.412+0000 7f1e6f7fe700 1 -- 172.21.15.43:0/2121259097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e68009b30 con 0x7f1e701592d0 2023-06-28T20:08:31.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.412+0000 7f1e6ffff700 1 --2- 172.21.15.43:0/2121259097 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e70071900 0x7f1e7014bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:31.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.412+0000 7f1e6f7fe700 1 --2- 172.21.15.43:0/2121259097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e701592d0 0x7f1e70158200 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1e6000c420 tx=0x7f1e6000c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:31.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.412+0000 7f1e6cff9700 1 -- 172.21.15.43:0/2121259097 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e600052d0 con 0x7f1e701592d0 2023-06-28T20:08:31.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.413+0000 7f1e6cff9700 1 -- 172.21.15.43:0/2121259097 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e60005430 con 0x7f1e701592d0 2023-06-28T20:08:31.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.413+0000 7f1e768ca700 1 -- 172.21.15.43:0/2121259097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e70157720 con 0x7f1e701592d0 2023-06-28T20:08:31.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.413+0000 7f1e6cff9700 1 -- 172.21.15.43:0/2121259097 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e600038c0 con 0x7f1e701592d0 2023-06-28T20:08:31.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.413+0000 7f1e768ca700 1 -- 172.21.15.43:0/2121259097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e700b3a30 con 0x7f1e701592d0 2023-06-28T20:08:31.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.413+0000 7f1e768ca700 1 -- 172.21.15.43:0/2121259097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e70157ee0 con 0x7f1e701592d0 2023-06-28T20:08:31.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.414+0000 7f1e6cff9700 1 -- 172.21.15.43:0/2121259097 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1e60003a20 con 0x7f1e701592d0 2023-06-28T20:08:31.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.414+0000 7f1e6cff9700 1 --2- 172.21.15.43:0/2121259097 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1e5006ace0 0x7f1e5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:31.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.414+0000 7f1e6cff9700 1 -- 172.21.15.43:0/2121259097 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1e60088820 con 0x7f1e701592d0 2023-06-28T20:08:31.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.414+0000 7f1e6ffff700 1 --2- 172.21.15.43:0/2121259097 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1e5006ace0 0x7f1e5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:31.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.415+0000 7f1e6ffff700 1 --2- 172.21.15.43:0/2121259097 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1e5006ace0 0x7f1e5006d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1e701f38d0 tx=0x7f1e58008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:31.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.419+0000 7f1e6cff9700 1 -- 172.21.15.43:0/2121259097 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1e600538c0 con 0x7f1e701592d0 2023-06-28T20:08:31.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.614+0000 7f1e768ca700 1 -- 172.21.15.43:0/2121259097 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1e700728b0 con 0x7f1e701592d0 2023-06-28T20:08:31.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.615+0000 7f1e6cff9700 1 -- 172.21.15.43:0/2121259097 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1e60056ee0 con 0x7f1e701592d0 2023-06-28T20:08:31.619 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-06-28T20:08:31.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.617+0000 7f1e768ca700 1 -- 172.21.15.43:0/2121259097 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1e5006ace0 msgr2=0x7f1e5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:31.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.617+0000 7f1e768ca700 1 --2- 172.21.15.43:0/2121259097 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1e5006ace0 0x7f1e5006d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1e701f38d0 tx=0x7f1e58008040 comp rx=0 tx=0).stop 2023-06-28T20:08:31.622 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.617+0000 7f1e768ca700 1 -- 172.21.15.43:0/2121259097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e701592d0 msgr2=0x7f1e70158200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:31.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.617+0000 7f1e768ca700 1 --2- 172.21.15.43:0/2121259097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e701592d0 0x7f1e70158200 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1e6000c420 tx=0x7f1e6000c730 comp rx=0 tx=0).stop 2023-06-28T20:08:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.617+0000 7f1e768ca700 1 -- 172.21.15.43:0/2121259097 shutdown_connections 2023-06-28T20:08:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.617+0000 7f1e768ca700 1 --2- 172.21.15.43:0/2121259097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e701596a0 0x7f1e700b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.617+0000 7f1e768ca700 1 --2- 172.21.15.43:0/2121259097 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1e5006ace0 0x7f1e5006d190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.617+0000 7f1e768ca700 1 --2- 172.21.15.43:0/2121259097 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1e70071900 0x7f1e7014bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.617+0000 7f1e768ca700 1 --2- 172.21.15.43:0/2121259097 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e701592d0 0x7f1e70158200 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.617+0000 7f1e768ca700 1 -- 172.21.15.43:0/2121259097 >> 172.21.15.43:0/2121259097 conn(0x7f1e7006b000 msgr2=0x7f1e7006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:31.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.617+0000 7f1e768ca700 1 -- 172.21.15.43:0/2121259097 shutdown_connections 2023-06-28T20:08:31.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.617+0000 7f1e768ca700 1 -- 172.21.15.43:0/2121259097 wait complete. 2023-06-28T20:08:31.636 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-28T20:08:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.758+0000 7f0f34a07700 1 Processor -- start 2023-06-28T20:08:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.758+0000 7f0f34a07700 1 -- start start 2023-06-28T20:08:31.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.759+0000 7f0f34a07700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0f3006d310 0x7f0f3006cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:31.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.759+0000 7f0f34a07700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f0f300cfe00 0x7f0f300cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:31.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.759+0000 7f0f34a07700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f300d0680 con 0x7f0f3015cae0 2023-06-28T20:08:31.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.759+0000 7f0f34a07700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f3006bb20 con 0x7f0f3006d310 2023-06-28T20:08:31.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.759+0000 7f0f34a07700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f3006be00 con 0x7f0f300cfe00 2023-06-28T20:08:31.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.759+0000 7f0f2e59c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0f3006d310 0x7f0f3006cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:31.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.760+0000 7f0f2e59c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0f3006d310 0x7f0f3006cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55244/0 (socket says 172.21.15.43:55244) 2023-06-28T20:08:31.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.760+0000 7f0f2e59c700 1 -- 172.21.15.43:0/3880691698 learned_addr learned my addr 172.21.15.43:0/3880691698 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:31.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.760+0000 7f0f2ed9d700 1 --2- 172.21.15.43:0/3880691698 >> v2:172.21.15.178:3300/0 conn(0x7f0f300cfe00 0x7f0f300cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:31.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.760+0000 7f0f2e59c700 1 -- 172.21.15.43:0/3880691698 >> v2:172.21.15.178:3300/0 conn(0x7f0f300cfe00 msgr2=0x7f0f300cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:31.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.760+0000 7f0f2e59c700 1 --2- 172.21.15.43:0/3880691698 >> v2:172.21.15.178:3300/0 conn(0x7f0f300cfe00 0x7f0f300cf7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.760+0000 7f0f2e59c700 1 -- 172.21.15.43:0/3880691698 >> v1:172.21.15.178:6789/0 conn(0x7f0f3015cae0 legacy=0x7f0f3006d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:31.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.760+0000 7f0f2e59c700 1 -- 172.21.15.43:0/3880691698 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3006c0e0 con 0x7f0f3006d310 2023-06-28T20:08:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.760+0000 7f0f2e59c700 1 --2- 172.21.15.43:0/3880691698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3006d310 0x7f0f3006cbc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0f300c46e0 tx=0x7f0f20009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa622d269d88cef server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.760+0000 7f0f2d59a700 1 -- 172.21.15.43:0/3880691698 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f20005e30 con 0x7f0f3006d310 2023-06-28T20:08:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.760+0000 7f0f2d59a700 1 -- 172.21.15.43:0/3880691698 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f20004030 con 0x7f0f3006d310 2023-06-28T20:08:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.760+0000 7f0f2d59a700 1 -- 172.21.15.43:0/3880691698 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f200043e0 con 0x7f0f3006d310 2023-06-28T20:08:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.761+0000 7f0f34a07700 1 -- 172.21.15.43:0/3880691698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3006d310 msgr2=0x7f0f3006cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.761+0000 7f0f34a07700 1 --2- 172.21.15.43:0/3880691698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3006d310 0x7f0f3006cbc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0f300c46e0 tx=0x7f0f20009bd0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.761+0000 7f0f34a07700 1 -- 172.21.15.43:0/3880691698 shutdown_connections 2023-06-28T20:08:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.761+0000 7f0f34a07700 1 --2- 172.21.15.43:0/3880691698 >> v2:172.21.15.178:3300/0 conn(0x7f0f300cfe00 0x7f0f300cf7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.761+0000 7f0f34a07700 1 --2- 172.21.15.43:0/3880691698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3006d310 0x7f0f3006cbc0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.761+0000 7f0f34a07700 1 -- 172.21.15.43:0/3880691698 >> 172.21.15.43:0/3880691698 conn(0x7f0f300664b0 msgr2=0x7f0f300d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.761+0000 7f0f34a07700 1 -- 172.21.15.43:0/3880691698 shutdown_connections 2023-06-28T20:08:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.761+0000 7f0f34a07700 1 -- 172.21.15.43:0/3880691698 wait complete. 2023-06-28T20:08:31.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.763+0000 7f0f34a07700 1 Processor -- start 2023-06-28T20:08:31.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.763+0000 7f0f34a07700 1 -- start start 2023-06-28T20:08:31.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.764+0000 7f0f34a07700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3006d310 0x7f0f301f6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:31.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.764+0000 7f0f34a07700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f300cfe00 0x7f0f300c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:31.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.764+0000 7f0f34a07700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f3015cae0 0x7f0f301f24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:31.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.764+0000 7f0f34a07700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f301f5ac0 con 0x7f0f3006d310 2023-06-28T20:08:31.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.764+0000 7f0f34a07700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f301f2fe0 con 0x7f0f3015cae0 2023-06-28T20:08:31.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.764+0000 7f0f34a07700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f301f32c0 con 0x7f0f300cfe00 2023-06-28T20:08:31.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.764+0000 7f0f2e59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3006d310 0x7f0f301f6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:31.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.764+0000 7f0f2e59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3006d310 0x7f0f301f6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55250/0 (socket says 172.21.15.43:55250) 2023-06-28T20:08:31.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.764+0000 7f0f2e59c700 1 -- 172.21.15.43:0/1684613512 learned_addr learned my addr 172.21.15.43:0/1684613512 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:31.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.764+0000 7f0f2dd9b700 1 --2- 172.21.15.43:0/1684613512 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f300cfe00 0x7f0f300c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:31.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.764+0000 7f0f2ed9d700 1 --2- 172.21.15.43:0/1684613512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f3015cae0 0x7f0f301f24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:31.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.764+0000 7f0f2e59c700 1 -- 172.21.15.43:0/1684613512 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f300cfe00 msgr2=0x7f0f300c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:31.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.764+0000 7f0f2e59c700 1 --2- 172.21.15.43:0/1684613512 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f300cfe00 0x7f0f300c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.764+0000 7f0f2e59c700 1 -- 172.21.15.43:0/1684613512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f3015cae0 msgr2=0x7f0f301f24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:31.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.764+0000 7f0f2e59c700 1 --2- 172.21.15.43:0/1684613512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f3015cae0 0x7f0f301f24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.764+0000 7f0f2e59c700 1 -- 172.21.15.43:0/1684613512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f20009500 con 0x7f0f3006d310 2023-06-28T20:08:31.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.764+0000 7f0f2ed9d700 1 --2- 172.21.15.43:0/1684613512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f3015cae0 0x7f0f301f24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:31.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.764+0000 7f0f2e59c700 1 --2- 172.21.15.43:0/1684613512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3006d310 0x7f0f301f6560 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0f20004000 tx=0x7f0f20004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:31.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.764+0000 7f0f1effd700 1 -- 172.21.15.43:0/1684613512 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f20003e10 con 0x7f0f3006d310 2023-06-28T20:08:31.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.764+0000 7f0f1effd700 1 -- 172.21.15.43:0/1684613512 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2001d430 con 0x7f0f3006d310 2023-06-28T20:08:31.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.764+0000 7f0f1effd700 1 -- 172.21.15.43:0/1684613512 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2001d7e0 con 0x7f0f3006d310 2023-06-28T20:08:31.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.764+0000 7f0f34a07700 1 -- 172.21.15.43:0/1684613512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f301f5d60 con 0x7f0f3006d310 2023-06-28T20:08:31.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.764+0000 7f0f34a07700 1 -- 172.21.15.43:0/1684613512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f300c72f0 con 0x7f0f3006d310 2023-06-28T20:08:31.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.765+0000 7f0f34a07700 1 -- 172.21.15.43:0/1684613512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f301f6280 con 0x7f0f3006d310 2023-06-28T20:08:31.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.766+0000 7f0f1effd700 1 -- 172.21.15.43:0/1684613512 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0f2001d940 con 0x7f0f3006d310 2023-06-28T20:08:31.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.766+0000 7f0f1effd700 1 --2- 172.21.15.43:0/1684613512 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0f1006ace0 0x7f0f1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:31.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.766+0000 7f0f1effd700 1 -- 172.21.15.43:0/1684613512 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0f2008a8c0 con 0x7f0f3006d310 2023-06-28T20:08:31.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.766+0000 7f0f2dd9b700 1 --2- 172.21.15.43:0/1684613512 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0f1006ace0 0x7f0f1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:31.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.767+0000 7f0f2dd9b700 1 --2- 172.21.15.43:0/1684613512 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0f1006ace0 0x7f0f1006d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0f301e74a0 tx=0x7f0f24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:31.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.771+0000 7f0f1effd700 1 -- 172.21.15.43:0/1684613512 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0f301f6280 con 0x7f0f3006d310 2023-06-28T20:08:31.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.985+0000 7f0f34a07700 1 -- 172.21.15.43:0/1684613512 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0f301f6280 con 0x7f0f3006d310 2023-06-28T20:08:31.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.986+0000 7f0f1effd700 1 -- 172.21.15.43:0/1684613512 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0f301f6280 con 0x7f0f3006d310 2023-06-28T20:08:31.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.988+0000 7f0f34a07700 1 -- 172.21.15.43:0/1684613512 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0f1006ace0 msgr2=0x7f0f1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:31.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.988+0000 7f0f34a07700 1 --2- 172.21.15.43:0/1684613512 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0f1006ace0 0x7f0f1006d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0f301e74a0 tx=0x7f0f24009040 comp rx=0 tx=0).stop 2023-06-28T20:08:31.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.988+0000 7f0f34a07700 1 -- 172.21.15.43:0/1684613512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3006d310 msgr2=0x7f0f301f6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:31.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.988+0000 7f0f34a07700 1 --2- 172.21.15.43:0/1684613512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3006d310 0x7f0f301f6560 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0f20004000 tx=0x7f0f20004750 comp rx=0 tx=0).stop 2023-06-28T20:08:31.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.988+0000 7f0f34a07700 1 -- 172.21.15.43:0/1684613512 shutdown_connections 2023-06-28T20:08:31.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.988+0000 7f0f34a07700 1 --2- 172.21.15.43:0/1684613512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f3015cae0 0x7f0f301f24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.988+0000 7f0f34a07700 1 --2- 172.21.15.43:0/1684613512 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f3006d310 0x7f0f301f6560 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.988+0000 7f0f34a07700 1 --2- 172.21.15.43:0/1684613512 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0f1006ace0 0x7f0f1006d190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.988+0000 7f0f34a07700 1 --2- 172.21.15.43:0/1684613512 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f300cfe00 0x7f0f300c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:31.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.988+0000 7f0f34a07700 1 -- 172.21.15.43:0/1684613512 >> 172.21.15.43:0/1684613512 conn(0x7f0f300664b0 msgr2=0x7f0f300af9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.988+0000 7f0f34a07700 1 -- 172.21.15.43:0/1684613512 shutdown_connections 2023-06-28T20:08:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:31.988+0000 7f0f34a07700 1 -- 172.21.15.43:0/1684613512 wait complete. 2023-06-28T20:08:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2023-06-28T20:08:32.004 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-06-28T20:08:32.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:32.005 DEBUG:teuthology.orchestra.run.smithi043:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-06-28T20:08:32.045 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2023-06-28T20:08:32.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:32.046 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-06-28T20:08:32.104 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:08:32.104 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:08:32.125 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:08:32.126 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:08:32.186 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:08:32.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.306+0000 7f0b90bfc700 1 Processor -- start 2023-06-28T20:08:32.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.306+0000 7f0b90bfc700 1 -- start start 2023-06-28T20:08:32.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.307+0000 7f0b90bfc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0b8c0d0630 0x7f0b8c0d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:32.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.307+0000 7f0b90bfc700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b8c0ccf10 con 0x7f0b8c0cbd40 2023-06-28T20:08:32.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.307+0000 7f0b90bfc700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b8c15cd80 con 0x7f0b8c06b710 2023-06-28T20:08:32.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.307+0000 7f0b90bfc700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b8c0d0a00 con 0x7f0b8c0d0630 2023-06-28T20:08:32.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.307+0000 7f0b8a59c700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0b8c0d0630 0x7f0b8c0d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:32.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.307+0000 7f0b89d9b700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0b8c0cbd40 0x7f0b8c06b000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44444/0 (socket says 172.21.15.43:44444) 2023-06-28T20:08:32.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.307+0000 7f0b89d9b700 1 -- 172.21.15.43:0/3062211530 learned_addr learned my addr 172.21.15.43:0/3062211530 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:32.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.308+0000 7f0b8a59c700 1 -- 172.21.15.43:0/3062211530 >> v1:172.21.15.178:6789/0 conn(0x7f0b8c06b710 legacy=0x7f0b8c0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:32.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.308+0000 7f0b8a59c700 1 -- 172.21.15.43:0/3062211530 >> v1:172.21.15.43:6789/0 conn(0x7f0b8c0cbd40 legacy=0x7f0b8c06b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:32.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.308+0000 7f0b8a59c700 1 -- 172.21.15.43:0/3062211530 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8c0cc300 con 0x7f0b8c0d0630 2023-06-28T20:08:32.313 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.308+0000 7f0b8959a700 1 -- 172.21.15.43:0/3062211530 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3908788630 0 0) 0x7f0b8c0ccf10 con 0x7f0b8c0cbd40 2023-06-28T20:08:32.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.308+0000 7f0b8a59c700 1 --2- 172.21.15.43:0/3062211530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b8c0d0630 0x7f0b8c0d1070 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009850 tx=0x7f0b80009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c273e64d857d8f server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:32.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.308+0000 7f0b8959a700 1 -- 172.21.15.43:0/3062211530 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b80005e30 con 0x7f0b8c0d0630 2023-06-28T20:08:32.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.308+0000 7f0b8959a700 1 -- 172.21.15.43:0/3062211530 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b80004030 con 0x7f0b8c0d0630 2023-06-28T20:08:32.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.308+0000 7f0b8959a700 1 -- 172.21.15.43:0/3062211530 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b800043c0 con 0x7f0b8c0d0630 2023-06-28T20:08:32.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.308+0000 7f0b90bfc700 1 -- 172.21.15.43:0/3062211530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b8c0d0630 msgr2=0x7f0b8c0d1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:32.314 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.308+0000 7f0b90bfc700 1 --2- 172.21.15.43:0/3062211530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b8c0d0630 0x7f0b8c0d1070 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009850 tx=0x7f0b80009b60 comp rx=0 tx=0).stop 2023-06-28T20:08:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.309+0000 7f0b90bfc700 1 -- 172.21.15.43:0/3062211530 shutdown_connections 2023-06-28T20:08:32.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.309+0000 7f0b90bfc700 1 --2- 172.21.15.43:0/3062211530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b8c0d0630 0x7f0b8c0d1070 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:32.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.309+0000 7f0b90bfc700 1 -- 172.21.15.43:0/3062211530 >> 172.21.15.43:0/3062211530 conn(0x7f0b8c0664b0 msgr2=0x7f0b8c09a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:32.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.309+0000 7f0b90bfc700 1 -- 172.21.15.43:0/3062211530 shutdown_connections 2023-06-28T20:08:32.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.309+0000 7f0b90bfc700 1 -- 172.21.15.43:0/3062211530 wait complete. 2023-06-28T20:08:32.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.310+0000 7f0b90bfc700 1 Processor -- start 2023-06-28T20:08:32.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.310+0000 7f0b90bfc700 1 -- start start 2023-06-28T20:08:32.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.311+0000 7f0b90bfc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b8c06b710 0x7f0b8c0c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:32.318 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.311+0000 7f0b90bfc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b8c0cbd40 0x7f0b8c1f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:32.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.311+0000 7f0b90bfc700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b8c0d0630 0x7f0b8c157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:32.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.311+0000 7f0b90bfc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c0c8d30 con 0x7f0b8c0cbd40 2023-06-28T20:08:32.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.311+0000 7f0b90bfc700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c0c9010 con 0x7f0b8c06b710 2023-06-28T20:08:32.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.311+0000 7f0b90bfc700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c0c92f0 con 0x7f0b8c0d0630 2023-06-28T20:08:32.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.311+0000 7f0b89d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b8c0cbd40 0x7f0b8c1f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:32.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.311+0000 7f0b89d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b8c0cbd40 0x7f0b8c1f20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55266/0 (socket says 172.21.15.43:55266) 2023-06-28T20:08:32.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.311+0000 7f0b89d9b700 1 -- 172.21.15.43:0/3195841561 learned_addr learned my addr 172.21.15.43:0/3195841561 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:32.319 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.311+0000 7f0b8a59c700 1 --2- 172.21.15.43:0/3195841561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b8c06b710 0x7f0b8c0c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:32.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.311+0000 7f0b8ad9d700 1 --2- 172.21.15.43:0/3195841561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b8c0d0630 0x7f0b8c157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:32.320 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.311+0000 7f0b89d9b700 1 -- 172.21.15.43:0/3195841561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b8c0d0630 msgr2=0x7f0b8c157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:32.324 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.311+0000 7f0b89d9b700 1 --2- 172.21.15.43:0/3195841561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b8c0d0630 0x7f0b8c157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:32.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.311+0000 7f0b89d9b700 1 -- 172.21.15.43:0/3195841561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b8c06b710 msgr2=0x7f0b8c0c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:32.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.311+0000 7f0b89d9b700 1 --2- 172.21.15.43:0/3195841561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b8c06b710 0x7f0b8c0c7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:32.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.311+0000 7f0b89d9b700 1 -- 172.21.15.43:0/3195841561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b80009430 con 0x7f0b8c0cbd40 2023-06-28T20:08:32.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.311+0000 7f0b8a59c700 1 --2- 172.21.15.43:0/3195841561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b8c06b710 0x7f0b8c0c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:32.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.311+0000 7f0b8ad9d700 1 --2- 172.21.15.43:0/3195841561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b8c0d0630 0x7f0b8c157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:32.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.311+0000 7f0b89d9b700 1 --2- 172.21.15.43:0/3195841561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b8c0cbd40 0x7f0b8c1f20b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00c520 tx=0x7f0b7c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:32.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.311+0000 7f0b7affd700 1 -- 172.21.15.43:0/3195841561 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c004c80 con 0x7f0b8c0cbd40 2023-06-28T20:08:32.325 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.311+0000 7f0b7affd700 1 -- 172.21.15.43:0/3195841561 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b7c004e00 con 0x7f0b8c0cbd40 2023-06-28T20:08:32.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.311+0000 7f0b90bfc700 1 -- 172.21.15.43:0/3195841561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b8c0c95d0 con 0x7f0b8c0cbd40 2023-06-28T20:08:32.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.311+0000 7f0b7affd700 1 -- 172.21.15.43:0/3195841561 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c007310 con 0x7f0b8c0cbd40 2023-06-28T20:08:32.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.312+0000 7f0b90bfc700 1 -- 172.21.15.43:0/3195841561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8c0c98b0 con 0x7f0b8c0cbd40 2023-06-28T20:08:32.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.312+0000 7f0b90bfc700 1 -- 172.21.15.43:0/3195841561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b8c0728b0 con 0x7f0b8c0cbd40 2023-06-28T20:08:32.326 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.313+0000 7f0b7affd700 1 -- 172.21.15.43:0/3195841561 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0b7c007470 con 0x7f0b8c0cbd40 2023-06-28T20:08:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.313+0000 7f0b7affd700 1 --2- 172.21.15.43:0/3195841561 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0b6c06ace0 0x7f0b6c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.313+0000 7f0b7affd700 1 -- 172.21.15.43:0/3195841561 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0b7c089170 con 0x7f0b8c0cbd40 2023-06-28T20:08:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.313+0000 7f0b8a59c700 1 --2- 172.21.15.43:0/3195841561 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0b6c06ace0 0x7f0b6c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.314+0000 7f0b8a59c700 1 --2- 172.21.15.43:0/3195841561 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0b6c06ace0 0x7f0b6c06d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009ba0 tx=0x7f0b80010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:32.328 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.318+0000 7f0b7affd700 1 -- 172.21.15.43:0/3195841561 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0b7c0542b0 con 0x7f0b8c0cbd40 2023-06-28T20:08:32.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.516+0000 7f0b90bfc700 1 -- 172.21.15.43:0/3195841561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b8c0cf880 con 0x7f0b8c0cbd40 2023-06-28T20:08:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.517+0000 7f0b7affd700 1 -- 172.21.15.43:0/3195841561 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+18293 (secure 0 0 0) 0x7f0b7c0578d0 con 0x7f0b8c0cbd40 2023-06-28T20:08:32.523 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:32.523 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":35,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:08:21.654904+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:08:17.811454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-06-28T20:08:19.046351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:08:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.519+0000 7f0b90bfc700 1 -- 172.21.15.43:0/3195841561 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0b6c06ace0 msgr2=0x7f0b6c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.519+0000 7f0b90bfc700 1 --2- 172.21.15.43:0/3195841561 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0b6c06ace0 0x7f0b6c06d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009ba0 tx=0x7f0b80010000 comp rx=0 tx=0).stop 2023-06-28T20:08:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.519+0000 7f0b90bfc700 1 -- 172.21.15.43:0/3195841561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b8c0cbd40 msgr2=0x7f0b8c1f20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.519+0000 7f0b90bfc700 1 --2- 172.21.15.43:0/3195841561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b8c0cbd40 0x7f0b8c1f20b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00c520 tx=0x7f0b7c00c8e0 comp rx=0 tx=0).stop 2023-06-28T20:08:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.519+0000 7f0b90bfc700 1 -- 172.21.15.43:0/3195841561 shutdown_connections 2023-06-28T20:08:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.519+0000 7f0b90bfc700 1 --2- 172.21.15.43:0/3195841561 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0b6c06ace0 0x7f0b6c06d190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.519+0000 7f0b90bfc700 1 --2- 172.21.15.43:0/3195841561 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0b8c0d0630 0x7f0b8c157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.519+0000 7f0b90bfc700 1 --2- 172.21.15.43:0/3195841561 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0b8c06b710 0x7f0b8c0c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.519+0000 7f0b90bfc700 1 --2- 172.21.15.43:0/3195841561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0b8c0cbd40 0x7f0b8c1f20b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.519+0000 7f0b90bfc700 1 -- 172.21.15.43:0/3195841561 >> 172.21.15.43:0/3195841561 conn(0x7f0b8c0664b0 msgr2=0x7f0b8c06f070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.519+0000 7f0b90bfc700 1 -- 172.21.15.43:0/3195841561 shutdown_connections 2023-06-28T20:08:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.519+0000 7f0b90bfc700 1 -- 172.21.15.43:0/3195841561 wait complete. 2023-06-28T20:08:32.535 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-28T20:07:13.580969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-28T20:07:14.038358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-28T20:08:17.811454+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-06-28T20:08:19.046351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-28T20:08:32.536 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-28T20:08:32.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.660+0000 7f4848058700 1 Processor -- start 2023-06-28T20:08:32.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.660+0000 7f4848058700 1 -- start start 2023-06-28T20:08:32.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.661+0000 7f4848058700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f484015c6c0 0x7f48400d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:32.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.661+0000 7f4848058700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48400d3bc0 0x7f48400d0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:32.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.661+0000 7f4848058700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48400b2ad0 con 0x7f48400b1e20 2023-06-28T20:08:32.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.661+0000 7f4848058700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48400b1c30 con 0x7f484015c6c0 2023-06-28T20:08:32.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.661+0000 7f4848058700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48400b1950 con 0x7f48400d3bc0 2023-06-28T20:08:32.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.661+0000 7f48465f5700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f48400b1e20 0x7f48400afb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44462/0 (socket says 172.21.15.43:44462) 2023-06-28T20:08:32.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.661+0000 7f48465f5700 1 -- 172.21.15.43:0/456121980 learned_addr learned my addr 172.21.15.43:0/456121980 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:32.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.661+0000 7f4845df4700 1 --2- 172.21.15.43:0/456121980 >> v2:172.21.15.178:3300/0 conn(0x7f484015c6c0 0x7f48400d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:32.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.661+0000 7f48455f3700 1 --2- 172.21.15.43:0/456121980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48400d3bc0 0x7f48400d0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:32.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.661+0000 7f4844df2700 1 -- 172.21.15.43:0/456121980 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2215241678 0 0) 0x7f48400b2ad0 con 0x7f48400b1e20 2023-06-28T20:08:32.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.662+0000 7f4844df2700 1 -- 172.21.15.43:0/456121980 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f482c0035b0 con 0x7f48400b1e20 2023-06-28T20:08:32.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.662+0000 7f4845df4700 1 -- 172.21.15.43:0/456121980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48400d3bc0 msgr2=0x7f48400d0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:32.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.662+0000 7f4845df4700 1 --2- 172.21.15.43:0/456121980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48400d3bc0 0x7f48400d0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:32.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.662+0000 7f4845df4700 1 -- 172.21.15.43:0/456121980 >> v1:172.21.15.43:6789/0 conn(0x7f48400b1e20 legacy=0x7f48400afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:32.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.662+0000 7f4845df4700 1 -- 172.21.15.43:0/456121980 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48400b2480 con 0x7f484015c6c0 2023-06-28T20:08:32.669 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.662+0000 7f4845df4700 1 --2- 172.21.15.43:0/456121980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f484015c6c0 0x7f48400d0ab0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4830009820 tx=0x7f4830009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2949c93031486586 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:32.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.662+0000 7f4844df2700 1 -- 172.21.15.43:0/456121980 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830005e30 con 0x7f484015c6c0 2023-06-28T20:08:32.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.662+0000 7f4844df2700 1 -- 172.21.15.43:0/456121980 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4830006030 con 0x7f484015c6c0 2023-06-28T20:08:32.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.662+0000 7f4844df2700 1 -- 172.21.15.43:0/456121980 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48300063a0 con 0x7f484015c6c0 2023-06-28T20:08:32.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.663+0000 7f4848058700 1 -- 172.21.15.43:0/456121980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f484015c6c0 msgr2=0x7f48400d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:32.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.663+0000 7f4848058700 1 --2- 172.21.15.43:0/456121980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f484015c6c0 0x7f48400d0ab0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4830009820 tx=0x7f4830009b30 comp rx=0 tx=0).stop 2023-06-28T20:08:32.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.663+0000 7f4848058700 1 -- 172.21.15.43:0/456121980 shutdown_connections 2023-06-28T20:08:32.670 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.663+0000 7f4848058700 1 --2- 172.21.15.43:0/456121980 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f484015c6c0 0x7f48400d0ab0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:32.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.663+0000 7f4848058700 1 --2- 172.21.15.43:0/456121980 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48400d3bc0 0x7f48400d0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:32.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.663+0000 7f4848058700 1 -- 172.21.15.43:0/456121980 >> 172.21.15.43:0/456121980 conn(0x7f484009c210 msgr2=0x7f48400d2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:32.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.663+0000 7f4848058700 1 -- 172.21.15.43:0/456121980 shutdown_connections 2023-06-28T20:08:32.671 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.663+0000 7f4848058700 1 -- 172.21.15.43:0/456121980 wait complete. 2023-06-28T20:08:32.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.664+0000 7f4848058700 1 Processor -- start 2023-06-28T20:08:32.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.664+0000 7f4848058700 1 -- start start 2023-06-28T20:08:32.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.665+0000 7f4848058700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48400b1e20 0x7f48400c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:32.673 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.665+0000 7f4848058700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48400d3bc0 0x7f4840152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:32.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.665+0000 7f4848058700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f484015c6c0 0x7f48401e3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:32.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.665+0000 7f4848058700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48401e5c20 con 0x7f484015c6c0 2023-06-28T20:08:32.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.665+0000 7f4848058700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48401e5f00 con 0x7f48400b1e20 2023-06-28T20:08:32.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.665+0000 7f4848058700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48401e61e0 con 0x7f48400d3bc0 2023-06-28T20:08:32.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.665+0000 7f48465f5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f484015c6c0 0x7f48401e3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:32.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.665+0000 7f48465f5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f484015c6c0 0x7f48401e3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55282/0 (socket says 172.21.15.43:55282) 2023-06-28T20:08:32.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.665+0000 7f48465f5700 1 -- 172.21.15.43:0/2414855175 learned_addr learned my addr 172.21.15.43:0/2414855175 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:32.674 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.665+0000 7f48455f3700 1 --2- 172.21.15.43:0/2414855175 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48400d3bc0 0x7f4840152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:32.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.665+0000 7f4845df4700 1 --2- 172.21.15.43:0/2414855175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48400b1e20 0x7f48400c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:32.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.665+0000 7f48465f5700 1 -- 172.21.15.43:0/2414855175 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48400d3bc0 msgr2=0x7f4840152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:32.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.665+0000 7f48465f5700 1 --2- 172.21.15.43:0/2414855175 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48400d3bc0 0x7f4840152620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:32.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.665+0000 7f48465f5700 1 -- 172.21.15.43:0/2414855175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48400b1e20 msgr2=0x7f48400c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:32.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.665+0000 7f48465f5700 1 --2- 172.21.15.43:0/2414855175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48400b1e20 0x7f48400c8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:32.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.665+0000 7f48465f5700 1 -- 172.21.15.43:0/2414855175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4830009430 con 0x7f484015c6c0 2023-06-28T20:08:32.675 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.665+0000 7f48465f5700 1 --2- 172.21.15.43:0/2414855175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f484015c6c0 0x7f48401e3cc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f483800caa0 tx=0x7f483800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:32.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.666+0000 7f48367fc700 1 -- 172.21.15.43:0/2414855175 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483800aa80 con 0x7f484015c6c0 2023-06-28T20:08:32.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.666+0000 7f48367fc700 1 -- 172.21.15.43:0/2414855175 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483800abe0 con 0x7f484015c6c0 2023-06-28T20:08:32.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.666+0000 7f4848058700 1 -- 172.21.15.43:0/2414855175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48401e64c0 con 0x7f484015c6c0 2023-06-28T20:08:32.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.666+0000 7f48367fc700 1 -- 172.21.15.43:0/2414855175 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4838012400 con 0x7f484015c6c0 2023-06-28T20:08:32.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.666+0000 7f4848058700 1 -- 172.21.15.43:0/2414855175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48401e4a10 con 0x7f484015c6c0 2023-06-28T20:08:32.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.667+0000 7f4848058700 1 -- 172.21.15.43:0/2414855175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48401e5d60 con 0x7f484015c6c0 2023-06-28T20:08:32.676 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.667+0000 7f48367fc700 1 -- 172.21.15.43:0/2414855175 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f483801b3b0 con 0x7f484015c6c0 2023-06-28T20:08:32.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.668+0000 7f48367fc700 1 --2- 172.21.15.43:0/2414855175 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f482406ace0 0x7f482406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:32.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.668+0000 7f4845df4700 1 --2- 172.21.15.43:0/2414855175 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f482406ace0 0x7f482406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:32.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.668+0000 7f48367fc700 1 -- 172.21.15.43:0/2414855175 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4838089250 con 0x7f484015c6c0 2023-06-28T20:08:32.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.668+0000 7f4845df4700 1 --2- 172.21.15.43:0/2414855175 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f482406ace0 0x7f482406d190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4830013040 tx=0x7f4830008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:32.677 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.672+0000 7f48367fc700 1 -- 172.21.15.43:0/2414855175 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f48401e5d60 con 0x7f484015c6c0 2023-06-28T20:08:32.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.868+0000 7f4848058700 1 -- 172.21.15.43:0/2414855175 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f48401e5d60 con 0x7f484015c6c0 2023-06-28T20:08:32.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.869+0000 7f48367fc700 1 -- 172.21.15.43:0/2414855175 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v35) v1 ==== 92+0+10 (secure 0 0 0) 0x7f48401e5d60 con 0x7f484015c6c0 2023-06-28T20:08:32.873 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-06-28T20:08:32.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.871+0000 7f4848058700 1 -- 172.21.15.43:0/2414855175 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f482406ace0 msgr2=0x7f482406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:32.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.871+0000 7f4848058700 1 --2- 172.21.15.43:0/2414855175 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f482406ace0 0x7f482406d190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4830013040 tx=0x7f4830008070 comp rx=0 tx=0).stop 2023-06-28T20:08:32.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.871+0000 7f4848058700 1 -- 172.21.15.43:0/2414855175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f484015c6c0 msgr2=0x7f48401e3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:32.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.871+0000 7f4848058700 1 --2- 172.21.15.43:0/2414855175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f484015c6c0 0x7f48401e3cc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f483800caa0 tx=0x7f483800ce60 comp rx=0 tx=0).stop 2023-06-28T20:08:32.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.871+0000 7f4848058700 1 -- 172.21.15.43:0/2414855175 shutdown_connections 2023-06-28T20:08:32.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.871+0000 7f4848058700 1 --2- 172.21.15.43:0/2414855175 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f484015c6c0 0x7f48401e3cc0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:32.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.871+0000 7f4848058700 1 --2- 172.21.15.43:0/2414855175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48400b1e20 0x7f48400c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:32.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.871+0000 7f4848058700 1 --2- 172.21.15.43:0/2414855175 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f482406ace0 0x7f482406d190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:32.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.871+0000 7f4848058700 1 --2- 172.21.15.43:0/2414855175 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f48400d3bc0 0x7f4840152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:32.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.871+0000 7f4848058700 1 -- 172.21.15.43:0/2414855175 >> 172.21.15.43:0/2414855175 conn(0x7f484009c210 msgr2=0x7f48400b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:32.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.871+0000 7f4848058700 1 -- 172.21.15.43:0/2414855175 shutdown_connections 2023-06-28T20:08:32.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:32.871+0000 7f4848058700 1 -- 172.21.15.43:0/2414855175 wait complete. 2023-06-28T20:08:32.887 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-28T20:08:33.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.011+0000 7f9214637700 1 Processor -- start 2023-06-28T20:08:33.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.011+0000 7f9214637700 1 -- start start 2023-06-28T20:08:33.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.012+0000 7f9214637700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f920c0aec90 0x7f920c160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:33.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.012+0000 7f9214637700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f920c06ccc0 0x7f920c0b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:33.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.012+0000 7f9214637700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920c160250 con 0x7f920c0af060 2023-06-28T20:08:33.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.012+0000 7f9214637700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f920c15cb30 con 0x7f920c06ccc0 2023-06-28T20:08:33.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.012+0000 7f9214637700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f920c071410 con 0x7f920c0aec90 2023-06-28T20:08:33.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.012+0000 7f9212bd4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f920c06ccc0 0x7f920c0b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:33.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.012+0000 7f9211bd2700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f920c0af060 0x7f920c070bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44472/0 (socket says 172.21.15.43:44472) 2023-06-28T20:08:33.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.012+0000 7f9211bd2700 1 -- 172.21.15.43:0/4163621673 learned_addr learned my addr 172.21.15.43:0/4163621673 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:33.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.012+0000 7f92123d3700 1 --2- 172.21.15.43:0/4163621673 >> v2:172.21.15.178:3300/0 conn(0x7f920c0aec90 0x7f920c160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:33.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.013+0000 7f9212bd4700 1 -- 172.21.15.43:0/4163621673 >> v2:172.21.15.178:3300/0 conn(0x7f920c0aec90 msgr2=0x7f920c160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:33.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.013+0000 7f9212bd4700 1 --2- 172.21.15.43:0/4163621673 >> v2:172.21.15.178:3300/0 conn(0x7f920c0aec90 0x7f920c160cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.013+0000 7f9212bd4700 1 -- 172.21.15.43:0/4163621673 >> v1:172.21.15.43:6789/0 conn(0x7f920c0af060 legacy=0x7f920c070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:33.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.013+0000 7f9212bd4700 1 -- 172.21.15.43:0/4163621673 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920c1604f0 con 0x7f920c06ccc0 2023-06-28T20:08:33.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.013+0000 7f92113d1700 1 -- 172.21.15.43:0/4163621673 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2685322998 0 0) 0x7f920c160250 con 0x7f920c0af060 2023-06-28T20:08:33.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.013+0000 7f9212bd4700 1 --2- 172.21.15.43:0/4163621673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f920c06ccc0 0x7f920c0b1880 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f920c06fce0 tx=0x7f920400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6987fbd5176b45c server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:33.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.013+0000 7f92113d1700 1 -- 172.21.15.43:0/4163621673 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920400e070 con 0x7f920c06ccc0 2023-06-28T20:08:33.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.013+0000 7f92113d1700 1 -- 172.21.15.43:0/4163621673 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920400bb20 con 0x7f920c06ccc0 2023-06-28T20:08:33.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.013+0000 7f92113d1700 1 -- 172.21.15.43:0/4163621673 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9204012430 con 0x7f920c06ccc0 2023-06-28T20:08:33.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.014+0000 7f9214637700 1 -- 172.21.15.43:0/4163621673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f920c06ccc0 msgr2=0x7f920c0b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:33.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.014+0000 7f9214637700 1 --2- 172.21.15.43:0/4163621673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f920c06ccc0 0x7f920c0b1880 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f920c06fce0 tx=0x7f920400b330 comp rx=0 tx=0).stop 2023-06-28T20:08:33.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.014+0000 7f9214637700 1 -- 172.21.15.43:0/4163621673 shutdown_connections 2023-06-28T20:08:33.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.014+0000 7f9214637700 1 --2- 172.21.15.43:0/4163621673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f920c06ccc0 0x7f920c0b1880 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.014+0000 7f9214637700 1 --2- 172.21.15.43:0/4163621673 >> v2:172.21.15.178:3300/0 conn(0x7f920c0aec90 0x7f920c160cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.014+0000 7f9214637700 1 -- 172.21.15.43:0/4163621673 >> 172.21.15.43:0/4163621673 conn(0x7f920c06b000 msgr2=0x7f920c06b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:33.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.014+0000 7f9214637700 1 -- 172.21.15.43:0/4163621673 shutdown_connections 2023-06-28T20:08:33.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.014+0000 7f9214637700 1 -- 172.21.15.43:0/4163621673 wait complete. 2023-06-28T20:08:33.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.015+0000 7f9214637700 1 Processor -- start 2023-06-28T20:08:33.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.015+0000 7f9214637700 1 -- start start 2023-06-28T20:08:33.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.016+0000 7f9214637700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f920c06ccc0 0x7f920c0acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:33.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.016+0000 7f9214637700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f920c0aec90 0x7f920c0ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:33.027 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.016+0000 7f9214637700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f920c0af060 0x7f920c1538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.016+0000 7f9214637700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c157ec0 con 0x7f920c0aec90 2023-06-28T20:08:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.016+0000 7f9214637700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920c1581a0 con 0x7f920c06ccc0 2023-06-28T20:08:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.016+0000 7f9214637700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920c158480 con 0x7f920c0af060 2023-06-28T20:08:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.016+0000 7f9211bd2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f920c0aec90 0x7f920c0ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.016+0000 7f9211bd2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f920c0aec90 0x7f920c0ad420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55294/0 (socket says 172.21.15.43:55294) 2023-06-28T20:08:33.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.016+0000 7f9211bd2700 1 -- 172.21.15.43:0/3179184596 learned_addr learned my addr 172.21.15.43:0/3179184596 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.016+0000 7f9212bd4700 1 --2- 172.21.15.43:0/3179184596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f920c0af060 0x7f920c1538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.016+0000 7f92123d3700 1 --2- 172.21.15.43:0/3179184596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f920c06ccc0 0x7f920c0acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.016+0000 7f9211bd2700 1 -- 172.21.15.43:0/3179184596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f920c0af060 msgr2=0x7f920c1538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.016+0000 7f9211bd2700 1 --2- 172.21.15.43:0/3179184596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f920c0af060 0x7f920c1538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.016+0000 7f9211bd2700 1 -- 172.21.15.43:0/3179184596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f920c06ccc0 msgr2=0x7f920c0acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.016+0000 7f9211bd2700 1 --2- 172.21.15.43:0/3179184596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f920c06ccc0 0x7f920c0acee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.016+0000 7f9211bd2700 1 -- 172.21.15.43:0/3179184596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9204009a70 con 0x7f920c0aec90 2023-06-28T20:08:33.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.016+0000 7f9211bd2700 1 --2- 172.21.15.43:0/3179184596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f920c0aec90 0x7f920c0ad420 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f920800c550 tx=0x7f920800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:33.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.016+0000 7f9202ffd700 1 -- 172.21.15.43:0/3179184596 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208004c80 con 0x7f920c0aec90 2023-06-28T20:08:33.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.016+0000 7f9202ffd700 1 -- 172.21.15.43:0/3179184596 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9208004e00 con 0x7f920c0aec90 2023-06-28T20:08:33.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.016+0000 7f9214637700 1 -- 172.21.15.43:0/3179184596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f920c158760 con 0x7f920c0aec90 2023-06-28T20:08:33.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.016+0000 7f9202ffd700 1 -- 172.21.15.43:0/3179184596 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208007310 con 0x7f920c0aec90 2023-06-28T20:08:33.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.017+0000 7f9214637700 1 -- 172.21.15.43:0/3179184596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f920c15f9e0 con 0x7f920c0aec90 2023-06-28T20:08:33.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.017+0000 7f9214637700 1 -- 172.21.15.43:0/3179184596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f920c158c80 con 0x7f920c0aec90 2023-06-28T20:08:33.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.018+0000 7f9202ffd700 1 -- 172.21.15.43:0/3179184596 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9208007470 con 0x7f920c0aec90 2023-06-28T20:08:33.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.018+0000 7f9202ffd700 1 --2- 172.21.15.43:0/3179184596 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f91f006ace0 0x7f91f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:33.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.018+0000 7f9202ffd700 1 -- 172.21.15.43:0/3179184596 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9208089190 con 0x7f920c0aec90 2023-06-28T20:08:33.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.018+0000 7f92123d3700 1 --2- 172.21.15.43:0/3179184596 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f91f006ace0 0x7f91f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:33.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.019+0000 7f92123d3700 1 --2- 172.21.15.43:0/3179184596 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f91f006ace0 0x7f91f006d190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f920c154480 tx=0x7f91fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:33.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.023+0000 7f9202ffd700 1 -- 172.21.15.43:0/3179184596 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f920c158c80 con 0x7f920c0aec90 2023-06-28T20:08:33.224 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.220+0000 7f9214637700 1 -- 172.21.15.43:0/3179184596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f920c158c80 con 0x7f920c0aec90 2023-06-28T20:08:33.225 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.220+0000 7f9202ffd700 1 -- 172.21.15.43:0/3179184596 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f920c158c80 con 0x7f920c0aec90 2023-06-28T20:08:33.225 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-06-28T20:08:33.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.222+0000 7f9214637700 1 -- 172.21.15.43:0/3179184596 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f91f006ace0 msgr2=0x7f91f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:33.227 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.222+0000 7f9214637700 1 --2- 172.21.15.43:0/3179184596 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f91f006ace0 0x7f91f006d190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f920c154480 tx=0x7f91fc008040 comp rx=0 tx=0).stop 2023-06-28T20:08:33.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.222+0000 7f9214637700 1 -- 172.21.15.43:0/3179184596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f920c0aec90 msgr2=0x7f920c0ad420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:33.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.222+0000 7f9214637700 1 --2- 172.21.15.43:0/3179184596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f920c0aec90 0x7f920c0ad420 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f920800c550 tx=0x7f920800c910 comp rx=0 tx=0).stop 2023-06-28T20:08:33.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.222+0000 7f9214637700 1 -- 172.21.15.43:0/3179184596 shutdown_connections 2023-06-28T20:08:33.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.222+0000 7f9214637700 1 --2- 172.21.15.43:0/3179184596 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f91f006ace0 0x7f91f006d190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.222+0000 7f9214637700 1 --2- 172.21.15.43:0/3179184596 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f920c0af060 0x7f920c1538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.222+0000 7f9214637700 1 --2- 172.21.15.43:0/3179184596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f920c06ccc0 0x7f920c0acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.222+0000 7f9214637700 1 --2- 172.21.15.43:0/3179184596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f920c0aec90 0x7f920c0ad420 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.229 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.222+0000 7f9214637700 1 -- 172.21.15.43:0/3179184596 >> 172.21.15.43:0/3179184596 conn(0x7f920c06b000 msgr2=0x7f920c0709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.222+0000 7f9214637700 1 -- 172.21.15.43:0/3179184596 shutdown_connections 2023-06-28T20:08:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.222+0000 7f9214637700 1 -- 172.21.15.43:0/3179184596 wait complete. 2023-06-28T20:08:33.239 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-28T20:08:33.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.363+0000 7f4f4d6bb700 1 Processor -- start 2023-06-28T20:08:33.368 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.363+0000 7f4f4d6bb700 1 -- start start 2023-06-28T20:08:33.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f4d6bb700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f4f480cf830 0x7f4f480cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:33.370 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f4d6bb700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f4815cf10 0x7f4f480b82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:33.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f4d6bb700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f480d0f00 con 0x7f4f480d0a80 2023-06-28T20:08:33.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f4d6bb700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f480d0140 con 0x7f4f480cf830 2023-06-28T20:08:33.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f4d6bb700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f480b8c80 con 0x7f4f4815cf10 2023-06-28T20:08:33.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f467fc700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4f480d0a80 0x7f4f480d1c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44488/0 (socket says 172.21.15.43:44488) 2023-06-28T20:08:33.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f467fc700 1 -- 172.21.15.43:0/2513470121 learned_addr learned my addr 172.21.15.43:0/2513470121 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:33.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f46ffd700 1 --2- 172.21.15.43:0/2513470121 >> v2:172.21.15.178:3300/0 conn(0x7f4f480cf830 0x7f4f480cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:33.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f477fe700 1 --2- 172.21.15.43:0/2513470121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f4815cf10 0x7f4f480b82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:33.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f45ffb700 1 -- 172.21.15.43:0/2513470121 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019832572 0 0) 0x7f4f480d0f00 con 0x7f4f480d0a80 2023-06-28T20:08:33.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f45ffb700 1 -- 172.21.15.43:0/2513470121 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f300035b0 con 0x7f4f480d0a80 2023-06-28T20:08:33.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f45ffb700 1 -- 172.21.15.43:0/2513470121 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1967870816 0 0) 0x7f4f300035b0 con 0x7f4f480d0a80 2023-06-28T20:08:33.372 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f45ffb700 1 -- 172.21.15.43:0/2513470121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f4815cf10 msgr2=0x7f4f480b82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f45ffb700 1 --2- 172.21.15.43:0/2513470121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f4815cf10 0x7f4f480b82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f45ffb700 1 -- 172.21.15.43:0/2513470121 >> v2:172.21.15.178:3300/0 conn(0x7f4f480cf830 msgr2=0x7f4f480cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f45ffb700 1 --2- 172.21.15.43:0/2513470121 >> v2:172.21.15.178:3300/0 conn(0x7f4f480cf830 0x7f4f480cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f45ffb700 1 -- 172.21.15.43:0/2513470121 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f480d11a0 con 0x7f4f480d0a80 2023-06-28T20:08:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f45ffb700 1 -- 172.21.15.43:0/2513470121 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f4f3c001670 con 0x7f4f480d0a80 2023-06-28T20:08:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f45ffb700 1 -- 172.21.15.43:0/2513470121 >> v1:172.21.15.43:6789/0 conn(0x7f4f480d0a80 legacy=0x7f4f480d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f45ffb700 1 --2- 172.21.15.43:0/2513470121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f4815cf10 0x7f4f30003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:33.373 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f45ffb700 1 --2- 172.21.15.43:0/2513470121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f30003f80 0x7f4f30004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f45ffb700 1 --2- 172.21.15.43:0/2513470121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f30004d90 0x7f4f30005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f45ffb700 1 -- 172.21.15.43:0/2513470121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f480d0140 con 0x7f4f30003f80 2023-06-28T20:08:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f45ffb700 1 -- 172.21.15.43:0/2513470121 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f480b8c80 con 0x7f4f30004d90 2023-06-28T20:08:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.364+0000 7f4f45ffb700 1 -- 172.21.15.43:0/2513470121 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3000ba10 con 0x7f4f4815cf10 2023-06-28T20:08:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.365+0000 7f4f467fc700 1 --2- 172.21.15.43:0/2513470121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f30003f80 0x7f4f30004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.365+0000 7f4f46ffd700 1 --2- 172.21.15.43:0/2513470121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f4815cf10 0x7f4f30003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.365+0000 7f4f477fe700 1 --2- 172.21.15.43:0/2513470121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f30004d90 0x7f4f30005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:33.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.365+0000 7f4f467fc700 1 -- 172.21.15.43:0/2513470121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f4815cf10 msgr2=0x7f4f30003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.365+0000 7f4f467fc700 1 --2- 172.21.15.43:0/2513470121 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f4815cf10 0x7f4f30003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.365+0000 7f4f467fc700 1 -- 172.21.15.43:0/2513470121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f30004d90 msgr2=0x7f4f30005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.365+0000 7f4f467fc700 1 --2- 172.21.15.43:0/2513470121 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f30004d90 0x7f4f30005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.365+0000 7f4f467fc700 1 -- 172.21.15.43:0/2513470121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f300035b0 con 0x7f4f30003f80 2023-06-28T20:08:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.365+0000 7f4f467fc700 1 --2- 172.21.15.43:0/2513470121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f30003f80 0x7f4f30004850 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4f480d0e50 tx=0x7f4f30005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303c9f56f71048ed server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.365+0000 7f4f45ffb700 1 -- 172.21.15.43:0/2513470121 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c007c60 con 0x7f4f30003f80 2023-06-28T20:08:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.365+0000 7f4f45ffb700 1 -- 172.21.15.43:0/2513470121 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3c00ce80 con 0x7f4f30003f80 2023-06-28T20:08:33.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.365+0000 7f4f45ffb700 1 -- 172.21.15.43:0/2513470121 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c005ba0 con 0x7f4f30003f80 2023-06-28T20:08:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.366+0000 7f4f4d6bb700 1 -- 172.21.15.43:0/2513470121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f30003f80 msgr2=0x7f4f30004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.366+0000 7f4f4d6bb700 1 --2- 172.21.15.43:0/2513470121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f30003f80 0x7f4f30004850 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4f480d0e50 tx=0x7f4f30005880 comp rx=0 tx=0).stop 2023-06-28T20:08:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.366+0000 7f4f477fe700 1 -- 172.21.15.43:0/2513470121 reap_dead start 2023-06-28T20:08:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.366+0000 7f4f4d6bb700 1 -- 172.21.15.43:0/2513470121 shutdown_connections 2023-06-28T20:08:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.366+0000 7f4f4d6bb700 1 -- 172.21.15.43:0/2513470121 >> 172.21.15.43:0/2513470121 conn(0x7f4f4806a640 msgr2=0x7f4f4806aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.366+0000 7f4f4d6bb700 1 -- 172.21.15.43:0/2513470121 shutdown_connections 2023-06-28T20:08:33.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.366+0000 7f4f4d6bb700 1 -- 172.21.15.43:0/2513470121 wait complete. 2023-06-28T20:08:33.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.367+0000 7f4f4d6bb700 1 Processor -- start 2023-06-28T20:08:33.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.367+0000 7f4f4d6bb700 1 -- start start 2023-06-28T20:08:33.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.368+0000 7f4f4d6bb700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f30003f80 0x7f4f48158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:33.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.368+0000 7f4f4d6bb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f4814f330 0x7f4f480b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:33.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.368+0000 7f4f4d6bb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f4814ee30 0x7f4f480b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:33.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.368+0000 7f4f4d6bb700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f481504b0 con 0x7f4f4814ee30 2023-06-28T20:08:33.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.368+0000 7f4f4d6bb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f480b3190 con 0x7f4f4814f330 2023-06-28T20:08:33.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.368+0000 7f4f4d6bb700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f48150bc0 con 0x7f4f30003f80 2023-06-28T20:08:33.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.368+0000 7f4f477fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f4814ee30 0x7f4f480b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:33.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.368+0000 7f4f477fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f4814ee30 0x7f4f480b71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55316/0 (socket says 172.21.15.43:55316) 2023-06-28T20:08:33.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.368+0000 7f4f477fe700 1 -- 172.21.15.43:0/1108335643 learned_addr learned my addr 172.21.15.43:0/1108335643 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:33.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.368+0000 7f4f467fc700 1 --2- 172.21.15.43:0/1108335643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f4814f330 0x7f4f480b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:33.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.368+0000 7f4f46ffd700 1 --2- 172.21.15.43:0/1108335643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f30003f80 0x7f4f48158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.368+0000 7f4f477fe700 1 -- 172.21.15.43:0/1108335643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f30003f80 msgr2=0x7f4f48158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.368+0000 7f4f477fe700 1 --2- 172.21.15.43:0/1108335643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f30003f80 0x7f4f48158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.368+0000 7f4f477fe700 1 -- 172.21.15.43:0/1108335643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f4814f330 msgr2=0x7f4f480b6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:33.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.368+0000 7f4f477fe700 1 --2- 172.21.15.43:0/1108335643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f4814f330 0x7f4f480b6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.368+0000 7f4f477fe700 1 -- 172.21.15.43:0/1108335643 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4814f8a0 con 0x7f4f4814ee30 2023-06-28T20:08:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.368+0000 7f4f467fc700 1 --2- 172.21.15.43:0/1108335643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f4814f330 0x7f4f480b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.368+0000 7f4f477fe700 1 --2- 172.21.15.43:0/1108335643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f4814ee30 0x7f4f480b71e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4f4808c160 tx=0x7f4f40011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.369+0000 7f4f2f7fe700 1 -- 172.21.15.43:0/1108335643 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4000e040 con 0x7f4f4814ee30 2023-06-28T20:08:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.369+0000 7f4f2f7fe700 1 -- 172.21.15.43:0/1108335643 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f4000f040 con 0x7f4f4814ee30 2023-06-28T20:08:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.369+0000 7f4f2f7fe700 1 -- 172.21.15.43:0/1108335643 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4000fa00 con 0x7f4f4814ee30 2023-06-28T20:08:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.369+0000 7f4f4d6bb700 1 -- 172.21.15.43:0/1108335643 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f4814fb80 con 0x7f4f4814ee30 2023-06-28T20:08:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.369+0000 7f4f4d6bb700 1 -- 172.21.15.43:0/1108335643 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f48150140 con 0x7f4f4814ee30 2023-06-28T20:08:33.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.369+0000 7f4f4d6bb700 1 -- 172.21.15.43:0/1108335643 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f481502d0 con 0x7f4f4814ee30 2023-06-28T20:08:33.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.370+0000 7f4f2f7fe700 1 -- 172.21.15.43:0/1108335643 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4f4000fb60 con 0x7f4f4814ee30 2023-06-28T20:08:33.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.370+0000 7f4f2f7fe700 1 --2- 172.21.15.43:0/1108335643 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f4f3406ace0 0x7f4f3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:33.384 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.371+0000 7f4f46ffd700 1 --2- 172.21.15.43:0/1108335643 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f4f3406ace0 0x7f4f3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:33.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.371+0000 7f4f2f7fe700 1 -- 172.21.15.43:0/1108335643 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4f40089a00 con 0x7f4f4814ee30 2023-06-28T20:08:33.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.371+0000 7f4f46ffd700 1 --2- 172.21.15.43:0/1108335643 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f4f3406ace0 0x7f4f3406d190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009ba0 tx=0x7f4f38008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:33.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.375+0000 7f4f2f7fe700 1 -- 172.21.15.43:0/1108335643 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4f40054ab0 con 0x7f4f4814ee30 2023-06-28T20:08:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.571+0000 7f4f4d6bb700 1 -- 172.21.15.43:0/1108335643 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4f480728b0 con 0x7f4f4814ee30 2023-06-28T20:08:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.572+0000 7f4f2f7fe700 1 -- 172.21.15.43:0/1108335643 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4f400580d0 con 0x7f4f4814ee30 2023-06-28T20:08:33.576 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-06-28T20:08:33.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.574+0000 7f4f4d6bb700 1 -- 172.21.15.43:0/1108335643 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f4f3406ace0 msgr2=0x7f4f3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:33.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.574+0000 7f4f4d6bb700 1 --2- 172.21.15.43:0/1108335643 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f4f3406ace0 0x7f4f3406d190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009ba0 tx=0x7f4f38008040 comp rx=0 tx=0).stop 2023-06-28T20:08:33.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.574+0000 7f4f4d6bb700 1 -- 172.21.15.43:0/1108335643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f4814ee30 msgr2=0x7f4f480b71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:33.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.574+0000 7f4f4d6bb700 1 --2- 172.21.15.43:0/1108335643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f4814ee30 0x7f4f480b71e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4f4808c160 tx=0x7f4f40011f80 comp rx=0 tx=0).stop 2023-06-28T20:08:33.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.574+0000 7f4f4d6bb700 1 -- 172.21.15.43:0/1108335643 shutdown_connections 2023-06-28T20:08:33.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.574+0000 7f4f4d6bb700 1 --2- 172.21.15.43:0/1108335643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f4814ee30 0x7f4f480b71e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.574+0000 7f4f4d6bb700 1 --2- 172.21.15.43:0/1108335643 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f4f3406ace0 0x7f4f3406d190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.574+0000 7f4f4d6bb700 1 --2- 172.21.15.43:0/1108335643 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f4f30003f80 0x7f4f48158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.574+0000 7f4f4d6bb700 1 --2- 172.21.15.43:0/1108335643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f4814f330 0x7f4f480b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.574+0000 7f4f4d6bb700 1 -- 172.21.15.43:0/1108335643 >> 172.21.15.43:0/1108335643 conn(0x7f4f4806a640 msgr2=0x7f4f480cd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:33.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.574+0000 7f4f4d6bb700 1 -- 172.21.15.43:0/1108335643 shutdown_connections 2023-06-28T20:08:33.581 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.574+0000 7f4f4d6bb700 1 -- 172.21.15.43:0/1108335643 wait complete. 2023-06-28T20:08:33.591 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-28T20:08:33.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.715+0000 7fea8cfce700 1 Processor -- start 2023-06-28T20:08:33.719 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.715+0000 7fea8cfce700 1 -- start start 2023-06-28T20:08:33.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.716+0000 7fea8cfce700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fea880cf6e0 0x7fea880ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:33.721 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.716+0000 7fea8cfce700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fea880b1920 0x7fea880b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:33.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.716+0000 7fea8cfce700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea880b2280 con 0x7fea880cca10 2023-06-28T20:08:33.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.716+0000 7fea8cfce700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea881609c0 con 0x7fea880b1920 2023-06-28T20:08:33.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.716+0000 7fea8cfce700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea88160e90 con 0x7fea880cf6e0 2023-06-28T20:08:33.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.716+0000 7fea85d9b700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fea880cca10 0x7fea88071270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44506/0 (socket says 172.21.15.43:44506) 2023-06-28T20:08:33.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.716+0000 7fea86d9d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fea880b1920 0x7fea880b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:33.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.716+0000 7fea85d9b700 1 -- 172.21.15.43:0/3548093678 learned_addr learned my addr 172.21.15.43:0/3548093678 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:33.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.716+0000 7fea8659c700 1 --2- 172.21.15.43:0/3548093678 >> v2:172.21.15.178:3300/0 conn(0x7fea880cf6e0 0x7fea880ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:33.722 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.716+0000 7fea8559a700 1 -- 172.21.15.43:0/3548093678 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248078918 0 0) 0x7fea880b2280 con 0x7fea880cca10 2023-06-28T20:08:33.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.717+0000 7fea8559a700 1 -- 172.21.15.43:0/3548093678 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea700035b0 con 0x7fea880cca10 2023-06-28T20:08:33.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.717+0000 7fea86d9d700 1 -- 172.21.15.43:0/3548093678 >> v2:172.21.15.178:3300/0 conn(0x7fea880cf6e0 msgr2=0x7fea880ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:33.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.717+0000 7fea86d9d700 1 --2- 172.21.15.43:0/3548093678 >> v2:172.21.15.178:3300/0 conn(0x7fea880cf6e0 0x7fea880ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.717+0000 7fea86d9d700 1 -- 172.21.15.43:0/3548093678 >> v1:172.21.15.43:6789/0 conn(0x7fea880cca10 legacy=0x7fea88071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:33.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.717+0000 7fea86d9d700 1 -- 172.21.15.43:0/3548093678 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea88160bb0 con 0x7fea880b1920 2023-06-28T20:08:33.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.717+0000 7fea86d9d700 1 --2- 172.21.15.43:0/3548093678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea880b1920 0x7fea880b1d40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fea880af480 tx=0x7fea8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89070cf7b7aebe24 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:33.723 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.717+0000 7fea8559a700 1 -- 172.21.15.43:0/3548093678 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8000e070 con 0x7fea880b1920 2023-06-28T20:08:33.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.717+0000 7fea8559a700 1 -- 172.21.15.43:0/3548093678 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8000bb20 con 0x7fea880b1920 2023-06-28T20:08:33.724 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.717+0000 7fea8559a700 1 -- 172.21.15.43:0/3548093678 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea80012430 con 0x7fea880b1920 2023-06-28T20:08:33.726 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.717+0000 7fea8cfce700 1 -- 172.21.15.43:0/3548093678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea880b1920 msgr2=0x7fea880b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:33.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.717+0000 7fea8cfce700 1 --2- 172.21.15.43:0/3548093678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea880b1920 0x7fea880b1d40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fea880af480 tx=0x7fea8000b330 comp rx=0 tx=0).stop 2023-06-28T20:08:33.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.718+0000 7fea8cfce700 1 -- 172.21.15.43:0/3548093678 shutdown_connections 2023-06-28T20:08:33.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.718+0000 7fea8cfce700 1 --2- 172.21.15.43:0/3548093678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea880b1920 0x7fea880b1d40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.718+0000 7fea8cfce700 1 --2- 172.21.15.43:0/3548093678 >> v2:172.21.15.178:3300/0 conn(0x7fea880cf6e0 0x7fea880ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.718+0000 7fea8cfce700 1 -- 172.21.15.43:0/3548093678 >> 172.21.15.43:0/3548093678 conn(0x7fea8806b000 msgr2=0x7fea8806b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:33.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.718+0000 7fea8cfce700 1 -- 172.21.15.43:0/3548093678 shutdown_connections 2023-06-28T20:08:33.727 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.718+0000 7fea8cfce700 1 -- 172.21.15.43:0/3548093678 wait complete. 2023-06-28T20:08:33.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.719+0000 7fea8cfce700 1 Processor -- start 2023-06-28T20:08:33.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.719+0000 7fea8cfce700 1 -- start start 2023-06-28T20:08:33.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.720+0000 7fea8cfce700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea880b1920 0x7fea880b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:33.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.720+0000 7fea8cfce700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea880cca10 0x7fea881f1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:33.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.720+0000 7fea8cfce700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea880cf6e0 0x7fea880c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:33.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.720+0000 7fea8cfce700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea880c3620 con 0x7fea880cf6e0 2023-06-28T20:08:33.728 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.720+0000 7fea8cfce700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8815e390 con 0x7fea880cca10 2023-06-28T20:08:33.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.720+0000 7fea8cfce700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea881f58f0 con 0x7fea880b1920 2023-06-28T20:08:33.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.721+0000 7fea86d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea880cf6e0 0x7fea880c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:33.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.721+0000 7fea86d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea880cf6e0 0x7fea880c5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55330/0 (socket says 172.21.15.43:55330) 2023-06-28T20:08:33.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.721+0000 7fea86d9d700 1 -- 172.21.15.43:0/878902020 learned_addr learned my addr 172.21.15.43:0/878902020 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:33.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.721+0000 7fea8659c700 1 --2- 172.21.15.43:0/878902020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea880b1920 0x7fea880b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:33.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.721+0000 7fea85d9b700 1 --2- 172.21.15.43:0/878902020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea880cca10 0x7fea881f1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:33.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.721+0000 7fea86d9d700 1 -- 172.21.15.43:0/878902020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea880b1920 msgr2=0x7fea880b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:33.729 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.721+0000 7fea86d9d700 1 --2- 172.21.15.43:0/878902020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea880b1920 0x7fea880b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.721+0000 7fea86d9d700 1 -- 172.21.15.43:0/878902020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea880cca10 msgr2=0x7fea881f1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:33.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.721+0000 7fea86d9d700 1 --2- 172.21.15.43:0/878902020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea880cca10 0x7fea881f1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.721+0000 7fea86d9d700 1 -- 172.21.15.43:0/878902020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea80009a70 con 0x7fea880cf6e0 2023-06-28T20:08:33.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.721+0000 7fea86d9d700 1 --2- 172.21.15.43:0/878902020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea880cf6e0 0x7fea880c5cb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fea80014040 tx=0x7fea80004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:33.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.721+0000 7fea76ffd700 1 -- 172.21.15.43:0/878902020 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea80012430 con 0x7fea880cf6e0 2023-06-28T20:08:33.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.721+0000 7fea76ffd700 1 -- 172.21.15.43:0/878902020 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea80020070 con 0x7fea880cf6e0 2023-06-28T20:08:33.730 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.721+0000 7fea8cfce700 1 -- 172.21.15.43:0/878902020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea881f5bd0 con 0x7fea880cf6e0 2023-06-28T20:08:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.721+0000 7fea76ffd700 1 -- 172.21.15.43:0/878902020 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea800098f0 con 0x7fea880cf6e0 2023-06-28T20:08:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.721+0000 7fea8cfce700 1 -- 172.21.15.43:0/878902020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea881f6190 con 0x7fea880cf6e0 2023-06-28T20:08:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.722+0000 7fea8cfce700 1 -- 172.21.15.43:0/878902020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea880c37d0 con 0x7fea880cf6e0 2023-06-28T20:08:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.723+0000 7fea76ffd700 1 -- 172.21.15.43:0/878902020 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fea80004040 con 0x7fea880cf6e0 2023-06-28T20:08:33.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.723+0000 7fea76ffd700 1 --2- 172.21.15.43:0/878902020 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fea6806ace0 0x7fea6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:33.732 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.723+0000 7fea76ffd700 1 -- 172.21.15.43:0/878902020 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fea80022070 con 0x7fea880cf6e0 2023-06-28T20:08:33.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.723+0000 7fea8659c700 1 --2- 172.21.15.43:0/878902020 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fea6806ace0 0x7fea6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:33.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.724+0000 7fea8659c700 1 --2- 172.21.15.43:0/878902020 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fea6806ace0 0x7fea6806d190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fea7c006b20 tx=0x7fea7c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:33.733 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.728+0000 7fea76ffd700 1 -- 172.21.15.43:0/878902020 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fea880c37d0 con 0x7fea880cf6e0 2023-06-28T20:08:33.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.925+0000 7fea8cfce700 1 -- 172.21.15.43:0/878902020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fea880728b0 con 0x7fea880cf6e0 2023-06-28T20:08:33.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.925+0000 7fea76ffd700 1 -- 172.21.15.43:0/878902020 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7fea80056330 con 0x7fea880cf6e0 2023-06-28T20:08:33.930 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-06-28T20:08:33.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.927+0000 7fea8cfce700 1 -- 172.21.15.43:0/878902020 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fea6806ace0 msgr2=0x7fea6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:33.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.927+0000 7fea8cfce700 1 --2- 172.21.15.43:0/878902020 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fea6806ace0 0x7fea6806d190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fea7c006b20 tx=0x7fea7c008040 comp rx=0 tx=0).stop 2023-06-28T20:08:33.931 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.927+0000 7fea8cfce700 1 -- 172.21.15.43:0/878902020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea880cf6e0 msgr2=0x7fea880c5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:33.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.927+0000 7fea8cfce700 1 --2- 172.21.15.43:0/878902020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea880cf6e0 0x7fea880c5cb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fea80014040 tx=0x7fea80004ad0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.927+0000 7fea8cfce700 1 -- 172.21.15.43:0/878902020 shutdown_connections 2023-06-28T20:08:33.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.927+0000 7fea8cfce700 1 --2- 172.21.15.43:0/878902020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea880cf6e0 0x7fea880c5cb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.927+0000 7fea8cfce700 1 --2- 172.21.15.43:0/878902020 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fea6806ace0 0x7fea6806d190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.927+0000 7fea8cfce700 1 --2- 172.21.15.43:0/878902020 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fea880b1920 0x7fea880b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.927+0000 7fea8cfce700 1 --2- 172.21.15.43:0/878902020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea880cca10 0x7fea881f1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:33.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.927+0000 7fea8cfce700 1 -- 172.21.15.43:0/878902020 >> 172.21.15.43:0/878902020 conn(0x7fea8806b000 msgr2=0x7fea880cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:33.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.927+0000 7fea8cfce700 1 -- 172.21.15.43:0/878902020 shutdown_connections 2023-06-28T20:08:33.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:33.927+0000 7fea8cfce700 1 -- 172.21.15.43:0/878902020 wait complete. 2023-06-28T20:08:33.943 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-28T20:08:34.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.066+0000 7f03d48e4700 1 Processor -- start 2023-06-28T20:08:34.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.066+0000 7f03d48e4700 1 -- start start 2023-06-28T20:08:34.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.067+0000 7f03d48e4700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f03d00aec90 0x7f03d0160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:34.079 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.067+0000 7f03d48e4700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d0160250 con 0x7f03d006ccc0 2023-06-28T20:08:34.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.067+0000 7f03d48e4700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d0071410 con 0x7f03d00af060 2023-06-28T20:08:34.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.067+0000 7f03d48e4700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d015cb30 con 0x7f03d00aec90 2023-06-28T20:08:34.080 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.067+0000 7f03ced9d700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f03d006ccc0 0x7f03d00b1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44522/0 (socket says 172.21.15.43:44522) 2023-06-28T20:08:34.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.067+0000 7f03ced9d700 1 -- 172.21.15.43:0/3262127422 learned_addr learned my addr 172.21.15.43:0/3262127422 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:34.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.068+0000 7f03ce59c700 1 --2- 172.21.15.43:0/3262127422 >> v2:172.21.15.178:3300/0 conn(0x7f03d00aec90 0x7f03d0160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:34.081 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.068+0000 7f03cd59a700 1 -- 172.21.15.43:0/3262127422 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3004279994 0 0) 0x7f03d0160250 con 0x7f03d006ccc0 2023-06-28T20:08:34.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.068+0000 7f03cd59a700 1 -- 172.21.15.43:0/3262127422 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03b80035b0 con 0x7f03d006ccc0 2023-06-28T20:08:34.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.068+0000 7f03ce59c700 1 -- 172.21.15.43:0/3262127422 >> v1:172.21.15.178:6789/0 conn(0x7f03d00af060 legacy=0x7f03d0070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:34.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.068+0000 7f03ce59c700 1 -- 172.21.15.43:0/3262127422 >> v1:172.21.15.43:6789/0 conn(0x7f03d006ccc0 legacy=0x7f03d00b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:34.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.068+0000 7f03cd59a700 1 -- 172.21.15.43:0/3262127422 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2808283763 0 0) 0x7f03b80035b0 con 0x7f03d006ccc0 2023-06-28T20:08:34.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.068+0000 7f03ce59c700 1 -- 172.21.15.43:0/3262127422 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d01604f0 con 0x7f03d00aec90 2023-06-28T20:08:34.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.068+0000 7f03ce59c700 1 --2- 172.21.15.43:0/3262127422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03d00aec90 0x7f03d0160cf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009840 tx=0x7f03c0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=558837c06de8fd7a server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:34.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.068+0000 7f03cd59a700 1 -- 172.21.15.43:0/3262127422 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c80019f0 con 0x7f03d00aec90 2023-06-28T20:08:34.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.068+0000 7f03cd59a700 1 -- 172.21.15.43:0/3262127422 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c0004030 con 0x7f03d00aec90 2023-06-28T20:08:34.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.068+0000 7f03cd59a700 1 -- 172.21.15.43:0/3262127422 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c00043c0 con 0x7f03d00aec90 2023-06-28T20:08:34.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.069+0000 7f03d48e4700 1 -- 172.21.15.43:0/3262127422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03d00aec90 msgr2=0x7f03d0160cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:34.083 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.069+0000 7f03d48e4700 1 --2- 172.21.15.43:0/3262127422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03d00aec90 0x7f03d0160cf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009840 tx=0x7f03c0009b50 comp rx=0 tx=0).stop 2023-06-28T20:08:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.069+0000 7f03d48e4700 1 -- 172.21.15.43:0/3262127422 shutdown_connections 2023-06-28T20:08:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.069+0000 7f03d48e4700 1 --2- 172.21.15.43:0/3262127422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03d00aec90 0x7f03d0160cf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.069+0000 7f03d48e4700 1 -- 172.21.15.43:0/3262127422 >> 172.21.15.43:0/3262127422 conn(0x7f03d006b000 msgr2=0x7f03d006b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.069+0000 7f03d48e4700 1 -- 172.21.15.43:0/3262127422 shutdown_connections 2023-06-28T20:08:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.069+0000 7f03d48e4700 1 -- 172.21.15.43:0/3262127422 wait complete. 2023-06-28T20:08:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.070+0000 7f03d48e4700 1 Processor -- start 2023-06-28T20:08:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.070+0000 7f03d48e4700 1 -- start start 2023-06-28T20:08:34.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.071+0000 7f03d48e4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d006ccc0 0x7f03d00c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:34.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.071+0000 7f03d48e4700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f03d00aec90 0x7f03d00c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:34.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.071+0000 7f03d48e4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03d00af060 0x7f03d01f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:34.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.071+0000 7f03d48e4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d00c4660 con 0x7f03d006ccc0 2023-06-28T20:08:34.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.071+0000 7f03d48e4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d00c9860 con 0x7f03d00af060 2023-06-28T20:08:34.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.071+0000 7f03d48e4700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d01f58f0 con 0x7f03d00aec90 2023-06-28T20:08:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.071+0000 7f03ce59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d006ccc0 0x7f03d00c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.071+0000 7f03ce59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d006ccc0 0x7f03d00c75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55340/0 (socket says 172.21.15.43:55340) 2023-06-28T20:08:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.071+0000 7f03ce59c700 1 -- 172.21.15.43:0/1019511756 learned_addr learned my addr 172.21.15.43:0/1019511756 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.071+0000 7f03ced9d700 1 --2- 172.21.15.43:0/1019511756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03d00af060 0x7f03d01f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.071+0000 7f03cdd9b700 1 --2- 172.21.15.43:0/1019511756 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f03d00aec90 0x7f03d00c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.072+0000 7f03ce59c700 1 -- 172.21.15.43:0/1019511756 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f03d00aec90 msgr2=0x7f03d00c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.072+0000 7f03ce59c700 1 --2- 172.21.15.43:0/1019511756 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f03d00aec90 0x7f03d00c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.072+0000 7f03ce59c700 1 -- 172.21.15.43:0/1019511756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03d00af060 msgr2=0x7f03d01f53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.072+0000 7f03ce59c700 1 --2- 172.21.15.43:0/1019511756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03d00af060 0x7f03d01f53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.072+0000 7f03ce59c700 1 -- 172.21.15.43:0/1019511756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c00094a0 con 0x7f03d006ccc0 2023-06-28T20:08:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.072+0000 7f03ced9d700 1 --2- 172.21.15.43:0/1019511756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03d00af060 0x7f03d01f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.072+0000 7f03ce59c700 1 --2- 172.21.15.43:0/1019511756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d006ccc0 0x7f03d00c75d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f03c00090b0 tx=0x7f03c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.072+0000 7f03beffd700 1 -- 172.21.15.43:0/1019511756 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c0011830 con 0x7f03d006ccc0 2023-06-28T20:08:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.072+0000 7f03beffd700 1 -- 172.21.15.43:0/1019511756 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c001f070 con 0x7f03d006ccc0 2023-06-28T20:08:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.072+0000 7f03d48e4700 1 -- 172.21.15.43:0/1019511756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d00c7c40 con 0x7f03d006ccc0 2023-06-28T20:08:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.072+0000 7f03beffd700 1 -- 172.21.15.43:0/1019511756 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c0011bd0 con 0x7f03d006ccc0 2023-06-28T20:08:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.072+0000 7f03d48e4700 1 -- 172.21.15.43:0/1019511756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d00c8200 con 0x7f03d006ccc0 2023-06-28T20:08:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.073+0000 7f03d48e4700 1 -- 172.21.15.43:0/1019511756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d00728b0 con 0x7f03d006ccc0 2023-06-28T20:08:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.073+0000 7f03beffd700 1 -- 172.21.15.43:0/1019511756 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f03c0011d30 con 0x7f03d006ccc0 2023-06-28T20:08:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.074+0000 7f03beffd700 1 --2- 172.21.15.43:0/1019511756 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f03b006ace0 0x7f03b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.074+0000 7f03cdd9b700 1 --2- 172.21.15.43:0/1019511756 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f03b006ace0 0x7f03b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.074+0000 7f03beffd700 1 -- 172.21.15.43:0/1019511756 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f03c0021070 con 0x7f03d006ccc0 2023-06-28T20:08:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.074+0000 7f03cdd9b700 1 --2- 172.21.15.43:0/1019511756 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f03b006ace0 0x7f03b006d190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f03d00b5a20 tx=0x7f03c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:34.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.079+0000 7f03beffd700 1 -- 172.21.15.43:0/1019511756 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f03c0056090 con 0x7f03d006ccc0 2023-06-28T20:08:34.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.295+0000 7f03d48e4700 1 -- 172.21.15.43:0/1019511756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f03d006d9c0 con 0x7f03d006ccc0 2023-06-28T20:08:34.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.295+0000 7f03beffd700 1 -- 172.21.15.43:0/1019511756 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) v1 ==== 80+0+0 (secure 0 0 0) 0x7f03c00596b0 con 0x7f03d006ccc0 2023-06-28T20:08:34.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.297+0000 7f03d48e4700 1 -- 172.21.15.43:0/1019511756 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f03b006ace0 msgr2=0x7f03b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:34.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.297+0000 7f03d48e4700 1 --2- 172.21.15.43:0/1019511756 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f03b006ace0 0x7f03b006d190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f03d00b5a20 tx=0x7f03c4009040 comp rx=0 tx=0).stop 2023-06-28T20:08:34.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.297+0000 7f03d48e4700 1 -- 172.21.15.43:0/1019511756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d006ccc0 msgr2=0x7f03d00c75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:34.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.297+0000 7f03d48e4700 1 --2- 172.21.15.43:0/1019511756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d006ccc0 0x7f03d00c75d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f03c00090b0 tx=0x7f03c0004750 comp rx=0 tx=0).stop 2023-06-28T20:08:34.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.297+0000 7f03d48e4700 1 -- 172.21.15.43:0/1019511756 shutdown_connections 2023-06-28T20:08:34.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.297+0000 7f03d48e4700 1 --2- 172.21.15.43:0/1019511756 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03d00af060 0x7f03d01f53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:34.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.297+0000 7f03d48e4700 1 --2- 172.21.15.43:0/1019511756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d006ccc0 0x7f03d00c75d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:34.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.297+0000 7f03d48e4700 1 --2- 172.21.15.43:0/1019511756 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f03b006ace0 0x7f03b006d190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:34.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.297+0000 7f03d48e4700 1 --2- 172.21.15.43:0/1019511756 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f03d00aec90 0x7f03d00c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:34.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.297+0000 7f03d48e4700 1 -- 172.21.15.43:0/1019511756 >> 172.21.15.43:0/1019511756 conn(0x7f03d006b000 msgr2=0x7f03d00b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:34.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.298+0000 7f03d48e4700 1 -- 172.21.15.43:0/1019511756 shutdown_connections 2023-06-28T20:08:34.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.298+0000 7f03d48e4700 1 -- 172.21.15.43:0/1019511756 wait complete. 2023-06-28T20:08:34.304 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2023-06-28T20:08:34.313 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-06-28T20:08:34.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:34.314 DEBUG:teuthology.orchestra.run.smithi178:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-06-28T20:08:34.373 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi178.front.sepia.ceph.com 2023-06-28T20:08:34.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:34.374 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-06-28T20:08:34.393 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:08:34.394 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:08:34.412 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13062 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-06-28T20:08:34.413 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:08:34.413 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:08:34.457 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3625 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-06-28T20:08:34.458 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-28T20:08:34.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.582+0000 7fa573ee3700 1 Processor -- start 2023-06-28T20:08:34.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.583+0000 7fa573ee3700 1 -- start start 2023-06-28T20:08:34.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.583+0000 7fa573ee3700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fa56c0af060 0x7fa56c0cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:34.590 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.583+0000 7fa573ee3700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa56c071410 con 0x7fa56c06a600 2023-06-28T20:08:34.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.583+0000 7fa573ee3700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa56c0716b0 con 0x7fa56c06ca80 2023-06-28T20:08:34.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.583+0000 7fa573ee3700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56c071990 con 0x7fa56c0af060 2023-06-28T20:08:34.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.584+0000 7fa572480700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa56c06a600 0x7fa56c0ac870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44538/0 (socket says 172.21.15.43:44538) 2023-06-28T20:08:34.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.584+0000 7fa572480700 1 -- 172.21.15.43:0/3407589300 learned_addr learned my addr 172.21.15.43:0/3407589300 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:34.591 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.584+0000 7fa570c7d700 1 -- 172.21.15.43:0/3407589300 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2104635932 0 0) 0x7fa56c071410 con 0x7fa56c06a600 2023-06-28T20:08:34.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.584+0000 7fa570c7d700 1 -- 172.21.15.43:0/3407589300 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5580035b0 con 0x7fa56c06a600 2023-06-28T20:08:34.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.584+0000 7fa571c7f700 1 --2- 172.21.15.43:0/3407589300 >> v2:172.21.15.178:3300/0 conn(0x7fa56c0af060 0x7fa56c0cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:34.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.584+0000 7fa570c7d700 1 -- 172.21.15.43:0/3407589300 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3805916080 0 0) 0x7fa5580035b0 con 0x7fa56c06a600 2023-06-28T20:08:34.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.584+0000 7fa570c7d700 1 -- 172.21.15.43:0/3407589300 >> v2:172.21.15.178:3300/0 conn(0x7fa56c0af060 msgr2=0x7fa56c0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:34.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.584+0000 7fa570c7d700 1 --2- 172.21.15.43:0/3407589300 >> v2:172.21.15.178:3300/0 conn(0x7fa56c0af060 0x7fa56c0cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:34.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.584+0000 7fa570c7d700 1 -- 172.21.15.43:0/3407589300 >> v1:172.21.15.178:6789/0 conn(0x7fa56c06ca80 legacy=0x7fa56c069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:34.592 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.584+0000 7fa570c7d700 1 -- 172.21.15.43:0/3407589300 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c0af430 con 0x7fa56c06a600 2023-06-28T20:08:34.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.584+0000 7fa570c7d700 1 -- 172.21.15.43:0/3407589300 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7fa5640019f0 con 0x7fa56c06a600 2023-06-28T20:08:34.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.584+0000 7fa570c7d700 1 -- 172.21.15.43:0/3407589300 >> v1:172.21.15.43:6789/0 conn(0x7fa56c06a600 legacy=0x7fa56c0ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:34.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.584+0000 7fa570c7d700 1 --2- 172.21.15.43:0/3407589300 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa558003a00 0x7fa558003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:34.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.584+0000 7fa570c7d700 1 --2- 172.21.15.43:0/3407589300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558004850 0x7fa558004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:34.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.584+0000 7fa570c7d700 1 --2- 172.21.15.43:0/3407589300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa558005200 0x7fa55800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:34.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.584+0000 7fa570c7d700 1 -- 172.21.15.43:0/3407589300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c071990 con 0x7fa558004850 2023-06-28T20:08:34.593 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.584+0000 7fa570c7d700 1 -- 172.21.15.43:0/3407589300 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55800c110 con 0x7fa558005200 2023-06-28T20:08:34.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.584+0000 7fa570c7d700 1 -- 172.21.15.43:0/3407589300 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55800c2d0 con 0x7fa558003a00 2023-06-28T20:08:34.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.584+0000 7fa57147e700 1 --2- 172.21.15.43:0/3407589300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558004850 0x7fa558004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:34.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.584+0000 7fa572480700 1 --2- 172.21.15.43:0/3407589300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa558005200 0x7fa55800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:34.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.584+0000 7fa571c7f700 1 --2- 172.21.15.43:0/3407589300 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa558003a00 0x7fa558003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:34.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.585+0000 7fa57147e700 1 -- 172.21.15.43:0/3407589300 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa558003a00 msgr2=0x7fa558003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:34.594 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.585+0000 7fa57147e700 1 --2- 172.21.15.43:0/3407589300 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa558003a00 0x7fa558003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:34.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.585+0000 7fa57147e700 1 -- 172.21.15.43:0/3407589300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa558005200 msgr2=0x7fa55800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:34.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.585+0000 7fa57147e700 1 --2- 172.21.15.43:0/3407589300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa558005200 0x7fa55800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:34.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.585+0000 7fa57147e700 1 -- 172.21.15.43:0/3407589300 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5580035b0 con 0x7fa558004850 2023-06-28T20:08:34.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.585+0000 7fa572480700 1 -- 172.21.15.43:0/3407589300 reap_dead start 2023-06-28T20:08:34.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.585+0000 7fa57147e700 1 --2- 172.21.15.43:0/3407589300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558004850 0x7fa558004cc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa5600017c0 tx=0x7fa558005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3814fa2af2150e41 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:34.595 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.585+0000 7fa570c7d700 1 -- 172.21.15.43:0/3407589300 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5600052d0 con 0x7fa558004850 2023-06-28T20:08:34.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.585+0000 7fa570c7d700 1 -- 172.21.15.43:0/3407589300 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa560005450 con 0x7fa558004850 2023-06-28T20:08:34.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.585+0000 7fa570c7d700 1 -- 172.21.15.43:0/3407589300 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5600038c0 con 0x7fa558004850 2023-06-28T20:08:34.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.585+0000 7fa573ee3700 1 -- 172.21.15.43:0/3407589300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558004850 msgr2=0x7fa558004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:34.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.585+0000 7fa573ee3700 1 --2- 172.21.15.43:0/3407589300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558004850 0x7fa558004cc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa5600017c0 tx=0x7fa558005880 comp rx=0 tx=0).stop 2023-06-28T20:08:34.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.585+0000 7fa573ee3700 1 -- 172.21.15.43:0/3407589300 shutdown_connections 2023-06-28T20:08:34.596 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.585+0000 7fa573ee3700 1 --2- 172.21.15.43:0/3407589300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558004850 0x7fa558004cc0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:34.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.585+0000 7fa573ee3700 1 -- 172.21.15.43:0/3407589300 >> 172.21.15.43:0/3407589300 conn(0x7fa56c0b1720 msgr2=0x7fa56c0b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:34.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.586+0000 7fa573ee3700 1 -- 172.21.15.43:0/3407589300 shutdown_connections 2023-06-28T20:08:34.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.586+0000 7fa573ee3700 1 -- 172.21.15.43:0/3407589300 wait complete. 2023-06-28T20:08:34.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.587+0000 7fa573ee3700 1 Processor -- start 2023-06-28T20:08:34.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.587+0000 7fa573ee3700 1 -- start start 2023-06-28T20:08:34.599 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.588+0000 7fa573ee3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558004850 0x7fa56c0c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:34.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.588+0000 7fa573ee3700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa56c156600 0x7fa56c155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:34.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.588+0000 7fa573ee3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa56c0b4a50 0x7fa56c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:34.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.588+0000 7fa573ee3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c0b5430 con 0x7fa558004850 2023-06-28T20:08:34.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.588+0000 7fa573ee3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c0ca240 con 0x7fa56c0b4a50 2023-06-28T20:08:34.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.588+0000 7fa573ee3700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c0c8b10 con 0x7fa56c156600 2023-06-28T20:08:34.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.588+0000 7fa571c7f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558004850 0x7fa56c0c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:34.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.588+0000 7fa571c7f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558004850 0x7fa56c0c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55360/0 (socket says 172.21.15.43:55360) 2023-06-28T20:08:34.600 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.588+0000 7fa571c7f700 1 -- 172.21.15.43:0/2442818923 learned_addr learned my addr 172.21.15.43:0/2442818923 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:34.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.588+0000 7fa57147e700 1 --2- 172.21.15.43:0/2442818923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa56c156600 0x7fa56c155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:34.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.588+0000 7fa572480700 1 --2- 172.21.15.43:0/2442818923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa56c0b4a50 0x7fa56c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:34.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.588+0000 7fa571c7f700 1 -- 172.21.15.43:0/2442818923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa56c156600 msgr2=0x7fa56c155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:34.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.588+0000 7fa571c7f700 1 --2- 172.21.15.43:0/2442818923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa56c156600 0x7fa56c155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:34.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.588+0000 7fa571c7f700 1 -- 172.21.15.43:0/2442818923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa56c0b4a50 msgr2=0x7fa56c0b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:34.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.588+0000 7fa571c7f700 1 --2- 172.21.15.43:0/2442818923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa56c0b4a50 0x7fa56c0b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:34.601 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.588+0000 7fa571c7f700 1 -- 172.21.15.43:0/2442818923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c0c8df0 con 0x7fa558004850 2023-06-28T20:08:34.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.588+0000 7fa57147e700 1 --2- 172.21.15.43:0/2442818923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa56c156600 0x7fa56c155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:34.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.588+0000 7fa571c7f700 1 --2- 172.21.15.43:0/2442818923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558004850 0x7fa56c0c5ef0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa5680119a0 tx=0x7fa568011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:34.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.588+0000 7fa55e7fc700 1 -- 172.21.15.43:0/2442818923 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56800f940 con 0x7fa558004850 2023-06-28T20:08:34.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.588+0000 7fa55e7fc700 1 -- 172.21.15.43:0/2442818923 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56800faa0 con 0x7fa558004850 2023-06-28T20:08:34.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.588+0000 7fa573ee3700 1 -- 172.21.15.43:0/2442818923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa56c0c90d0 con 0x7fa558004850 2023-06-28T20:08:34.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.588+0000 7fa55e7fc700 1 -- 172.21.15.43:0/2442818923 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56800fe30 con 0x7fa558004850 2023-06-28T20:08:34.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.588+0000 7fa573ee3700 1 -- 172.21.15.43:0/2442818923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56c0c6430 con 0x7fa558004850 2023-06-28T20:08:34.602 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.589+0000 7fa573ee3700 1 -- 172.21.15.43:0/2442818923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa56c0728b0 con 0x7fa558004850 2023-06-28T20:08:34.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.590+0000 7fa55e7fc700 1 -- 172.21.15.43:0/2442818923 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa568014440 con 0x7fa558004850 2023-06-28T20:08:34.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.590+0000 7fa55e7fc700 1 --2- 172.21.15.43:0/2442818923 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa55006ace0 0x7fa55006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:34.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.590+0000 7fa57147e700 1 --2- 172.21.15.43:0/2442818923 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa55006ace0 0x7fa55006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:34.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.590+0000 7fa55e7fc700 1 -- 172.21.15.43:0/2442818923 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa568089bc0 con 0x7fa558004850 2023-06-28T20:08:34.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.591+0000 7fa57147e700 1 --2- 172.21.15.43:0/2442818923 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa55006ace0 0x7fa55006d190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa560000c00 tx=0x7fa560009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:34.603 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.595+0000 7fa55e7fc700 1 -- 172.21.15.43:0/2442818923 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa568058700 con 0x7fa558004850 2023-06-28T20:08:34.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.836+0000 7fa573ee3700 1 -- 172.21.15.43:0/2442818923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa56c06e690 con 0x7fa558004850 2023-06-28T20:08:34.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.836+0000 7fa55e7fc700 1 -- 172.21.15.43:0/2442818923 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4674 (secure 0 0 0) 0x7fa568058290 con 0x7fa558004850 2023-06-28T20:08:34.842 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:34.842 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1274138939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":1274138939},{"type":"v1","addr":"172.21.15.178:6835","nonce":1274138939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3882460026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3882460026},{"type":"v1","addr":"172.21.15.178:6834","nonce":3882460026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/382468709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":382468709},{"type":"v1","addr":"172.21.15.43:6836","nonce":382468709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-28T20:08:20.073546+0000","modified":"2023-06-28T20:08:23.914623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5112},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5112":{"gid":5112,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6837/1096784002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":1096784002},{"type":"v1","addr":"172.21.15.43:6837","nonce":1096784002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-06-28T20:08:34.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.838+0000 7fa573ee3700 1 -- 172.21.15.43:0/2442818923 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa55006ace0 msgr2=0x7fa55006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:34.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.838+0000 7fa573ee3700 1 --2- 172.21.15.43:0/2442818923 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa55006ace0 0x7fa55006d190 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa560000c00 tx=0x7fa560009030 comp rx=0 tx=0).stop 2023-06-28T20:08:34.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.838+0000 7fa573ee3700 1 -- 172.21.15.43:0/2442818923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558004850 msgr2=0x7fa56c0c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:34.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.838+0000 7fa573ee3700 1 --2- 172.21.15.43:0/2442818923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558004850 0x7fa56c0c5ef0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa5680119a0 tx=0x7fa568011d60 comp rx=0 tx=0).stop 2023-06-28T20:08:34.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.838+0000 7fa573ee3700 1 -- 172.21.15.43:0/2442818923 shutdown_connections 2023-06-28T20:08:34.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.838+0000 7fa573ee3700 1 --2- 172.21.15.43:0/2442818923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa56c0b4a50 0x7fa56c0b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:34.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.838+0000 7fa573ee3700 1 --2- 172.21.15.43:0/2442818923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558004850 0x7fa56c0c5ef0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:34.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.838+0000 7fa573ee3700 1 --2- 172.21.15.43:0/2442818923 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fa55006ace0 0x7fa55006d190 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:34.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.838+0000 7fa573ee3700 1 --2- 172.21.15.43:0/2442818923 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fa56c156600 0x7fa56c155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:34.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.838+0000 7fa573ee3700 1 -- 172.21.15.43:0/2442818923 >> 172.21.15.43:0/2442818923 conn(0x7fa56c0b1720 msgr2=0x7fa56c0ce820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:34.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.839+0000 7fa573ee3700 1 -- 172.21.15.43:0/2442818923 shutdown_connections 2023-06-28T20:08:34.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.839+0000 7fa573ee3700 1 -- 172.21.15.43:0/2442818923 wait complete. 2023-06-28T20:08:34.845 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 15 2023-06-28T20:08:34.856 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:08:34.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.979+0000 7f4252831700 1 Processor -- start 2023-06-28T20:08:34.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.979+0000 7f4252831700 1 -- start start 2023-06-28T20:08:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.980+0000 7f4252831700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f424c0cc600 0x7f424c15a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.980+0000 7f4252831700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f424c159820 0x7f424c0cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.980+0000 7f4252831700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f424c0ccf10 con 0x7f424c06ee20 2023-06-28T20:08:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.980+0000 7f4252831700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f424c0ce0a0 con 0x7f424c159820 2023-06-28T20:08:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.980+0000 7f4252831700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f424c0ce380 con 0x7f424c0cc600 2023-06-28T20:08:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.980+0000 7f424b7fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f424c159820 0x7f424c0cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.980+0000 7f4250dce700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f424c06ee20 0x7f424c0cd7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44558/0 (socket says 172.21.15.43:44558) 2023-06-28T20:08:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.980+0000 7f4250dce700 1 -- 172.21.15.43:0/2487009239 learned_addr learned my addr 172.21.15.43:0/2487009239 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.980+0000 7f424bfff700 1 --2- 172.21.15.43:0/2487009239 >> v2:172.21.15.178:3300/0 conn(0x7f424c0cc600 0x7f424c15a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.980+0000 7f424b7fe700 1 -- 172.21.15.43:0/2487009239 >> v2:172.21.15.178:3300/0 conn(0x7f424c0cc600 msgr2=0x7f424c15a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.980+0000 7f424b7fe700 1 --2- 172.21.15.43:0/2487009239 >> v2:172.21.15.178:3300/0 conn(0x7f424c0cc600 0x7f424c15a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.980+0000 7f424b7fe700 1 -- 172.21.15.43:0/2487009239 >> v1:172.21.15.43:6789/0 conn(0x7f424c06ee20 legacy=0x7f424c0cd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.980+0000 7f424b7fe700 1 -- 172.21.15.43:0/2487009239 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f424c159c90 con 0x7f424c159820 2023-06-28T20:08:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.980+0000 7f424affd700 1 -- 172.21.15.43:0/2487009239 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 462405859 0 0) 0x7f424c0ccf10 con 0x7f424c06ee20 2023-06-28T20:08:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.980+0000 7f424b7fe700 1 --2- 172.21.15.43:0/2487009239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f424c159820 0x7f424c0cc9d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f424c0720f0 tx=0x7f4240009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c06d575393649df server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.980+0000 7f424affd700 1 -- 172.21.15.43:0/2487009239 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240011830 con 0x7f424c159820 2023-06-28T20:08:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.980+0000 7f424affd700 1 -- 172.21.15.43:0/2487009239 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4240011990 con 0x7f424c159820 2023-06-28T20:08:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.980+0000 7f424affd700 1 -- 172.21.15.43:0/2487009239 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240011d20 con 0x7f424c159820 2023-06-28T20:08:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.981+0000 7f4252831700 1 -- 172.21.15.43:0/2487009239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f424c159820 msgr2=0x7f424c0cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.981+0000 7f4252831700 1 --2- 172.21.15.43:0/2487009239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f424c159820 0x7f424c0cc9d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f424c0720f0 tx=0x7f4240009b20 comp rx=0 tx=0).stop 2023-06-28T20:08:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.981+0000 7f4252831700 1 -- 172.21.15.43:0/2487009239 shutdown_connections 2023-06-28T20:08:34.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.981+0000 7f4252831700 1 --2- 172.21.15.43:0/2487009239 >> v2:172.21.15.178:3300/0 conn(0x7f424c0cc600 0x7f424c15a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:34.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.981+0000 7f4252831700 1 --2- 172.21.15.43:0/2487009239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f424c159820 0x7f424c0cc9d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:34.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.981+0000 7f4252831700 1 -- 172.21.15.43:0/2487009239 >> 172.21.15.43:0/2487009239 conn(0x7f424c0b1720 msgr2=0x7f424c0b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:34.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.981+0000 7f4252831700 1 -- 172.21.15.43:0/2487009239 shutdown_connections 2023-06-28T20:08:34.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.981+0000 7f4252831700 1 -- 172.21.15.43:0/2487009239 wait complete. 2023-06-28T20:08:34.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.982+0000 7f4252831700 1 Processor -- start 2023-06-28T20:08:34.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.982+0000 7f4252831700 1 -- start start 2023-06-28T20:08:34.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.983+0000 7f4252831700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f424c06ee20 0x7f424c0c6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:34.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.983+0000 7f4252831700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f424c0cc600 0x7f424c0cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:34.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.983+0000 7f4252831700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f424c159820 0x7f424c0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:34.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.983+0000 7f4252831700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424c0b43a0 con 0x7f424c159820 2023-06-28T20:08:34.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.983+0000 7f4252831700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424c1f5ba0 con 0x7f424c06ee20 2023-06-28T20:08:34.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.983+0000 7f4252831700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f424c1f5e80 con 0x7f424c0cc600 2023-06-28T20:08:34.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.983+0000 7f4250dce700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f424c159820 0x7f424c0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:34.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.983+0000 7f4250dce700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f424c159820 0x7f424c0cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55378/0 (socket says 172.21.15.43:55378) 2023-06-28T20:08:34.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.983+0000 7f4250dce700 1 -- 172.21.15.43:0/2249236022 learned_addr learned my addr 172.21.15.43:0/2249236022 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:34.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.983+0000 7f424bfff700 1 --2- 172.21.15.43:0/2249236022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f424c06ee20 0x7f424c0c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:34.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.983+0000 7f424b7fe700 1 --2- 172.21.15.43:0/2249236022 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f424c0cc600 0x7f424c0cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:34.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.984+0000 7f4250dce700 1 -- 172.21.15.43:0/2249236022 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f424c0cc600 msgr2=0x7f424c0cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:34.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.984+0000 7f4250dce700 1 --2- 172.21.15.43:0/2249236022 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f424c0cc600 0x7f424c0cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:34.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.984+0000 7f4250dce700 1 -- 172.21.15.43:0/2249236022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f424c06ee20 msgr2=0x7f424c0c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:34.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.984+0000 7f4250dce700 1 --2- 172.21.15.43:0/2249236022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f424c06ee20 0x7f424c0c6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:34.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.984+0000 7f4250dce700 1 -- 172.21.15.43:0/2249236022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42400094a0 con 0x7f424c159820 2023-06-28T20:08:34.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.984+0000 7f4250dce700 1 --2- 172.21.15.43:0/2249236022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f424c159820 0x7f424c0cb630 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4244002230 tx=0x7f424400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:34.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.984+0000 7f4248ff9700 1 -- 172.21.15.43:0/2249236022 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424400ac20 con 0x7f424c159820 2023-06-28T20:08:34.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.984+0000 7f4248ff9700 1 -- 172.21.15.43:0/2249236022 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424400ada0 con 0x7f424c159820 2023-06-28T20:08:34.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.984+0000 7f4252831700 1 -- 172.21.15.43:0/2249236022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f424c1f6160 con 0x7f424c159820 2023-06-28T20:08:34.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.984+0000 7f4248ff9700 1 -- 172.21.15.43:0/2249236022 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4244007310 con 0x7f424c159820 2023-06-28T20:08:34.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.984+0000 7f4252831700 1 -- 172.21.15.43:0/2249236022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f424c15af90 con 0x7f424c159820 2023-06-28T20:08:34.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.985+0000 7f4252831700 1 -- 172.21.15.43:0/2249236022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f424c1f5ce0 con 0x7f424c159820 2023-06-28T20:08:34.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.985+0000 7f4248ff9700 1 -- 172.21.15.43:0/2249236022 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4244007470 con 0x7f424c159820 2023-06-28T20:08:34.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.986+0000 7f4248ff9700 1 --2- 172.21.15.43:0/2249236022 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f423806ace0 0x7f423806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:34.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.986+0000 7f4248ff9700 1 -- 172.21.15.43:0/2249236022 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4244089410 con 0x7f424c159820 2023-06-28T20:08:34.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.986+0000 7f424bfff700 1 --2- 172.21.15.43:0/2249236022 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f423806ace0 0x7f423806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:34.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.986+0000 7f424bfff700 1 --2- 172.21.15.43:0/2249236022 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f423806ace0 0x7f423806d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f423c009930 tx=0x7f423c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:34.999 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:34.990+0000 7f4248ff9700 1 -- 172.21.15.43:0/2249236022 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f424c1f5ce0 con 0x7f424c159820 2023-06-28T20:08:35.193 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.189+0000 7f4252831700 1 -- 172.21.15.43:0/2249236022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f424c1f5ce0 con 0x7f424c159820 2023-06-28T20:08:35.194 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.190+0000 7f4248ff9700 1 -- 172.21.15.43:0/2249236022 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+18293 (secure 0 0 0) 0x7f424c1f5ce0 con 0x7f424c159820 2023-06-28T20:08:35.194 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:35.195 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":35,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:08:21.654904+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:08:17.811454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-06-28T20:08:19.046351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:08:35.196 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.192+0000 7f4252831700 1 -- 172.21.15.43:0/2249236022 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f423806ace0 msgr2=0x7f423806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:35.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.192+0000 7f4252831700 1 --2- 172.21.15.43:0/2249236022 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f423806ace0 0x7f423806d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f423c009930 tx=0x7f423c008040 comp rx=0 tx=0).stop 2023-06-28T20:08:35.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.192+0000 7f4252831700 1 -- 172.21.15.43:0/2249236022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f424c159820 msgr2=0x7f424c0cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:35.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.192+0000 7f4252831700 1 --2- 172.21.15.43:0/2249236022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f424c159820 0x7f424c0cb630 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4244002230 tx=0x7f424400a060 comp rx=0 tx=0).stop 2023-06-28T20:08:35.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.192+0000 7f4252831700 1 -- 172.21.15.43:0/2249236022 shutdown_connections 2023-06-28T20:08:35.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.192+0000 7f4252831700 1 --2- 172.21.15.43:0/2249236022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f424c159820 0x7f424c0cb630 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.192+0000 7f4252831700 1 --2- 172.21.15.43:0/2249236022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f424c06ee20 0x7f424c0c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.192+0000 7f4252831700 1 --2- 172.21.15.43:0/2249236022 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f423806ace0 0x7f423806d190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.192+0000 7f4252831700 1 --2- 172.21.15.43:0/2249236022 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f424c0cc600 0x7f424c0cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.192+0000 7f4252831700 1 -- 172.21.15.43:0/2249236022 >> 172.21.15.43:0/2249236022 conn(0x7f424c0b1720 msgr2=0x7f424c15d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:35.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.192+0000 7f4252831700 1 -- 172.21.15.43:0/2249236022 shutdown_connections 2023-06-28T20:08:35.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.192+0000 7f4252831700 1 -- 172.21.15.43:0/2249236022 wait complete. 2023-06-28T20:08:35.209 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-06-28T20:07:13.580969+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-06-28T20:07:14.038358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-28T20:08:17.811454+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-06-28T20:08:19.046351+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-28T20:08:35.210 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-28T20:08:35.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.334+0000 7f2f47a75700 1 Processor -- start 2023-06-28T20:08:35.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.334+0000 7f2f47a75700 1 -- start start 2023-06-28T20:08:35.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.335+0000 7f2f47a75700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f2f401592d0 0x7f2f400b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:35.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.335+0000 7f2f47a75700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f40070b40 con 0x7f2f401596a0 2023-06-28T20:08:35.341 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.335+0000 7f2f47a75700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f40070e20 con 0x7f2f40071900 2023-06-28T20:08:35.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.335+0000 7f2f47a75700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f40071100 con 0x7f2f401592d0 2023-06-28T20:08:35.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.335+0000 7f2f45010700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2f401596a0 0x7f2f4015d070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44568/0 (socket says 172.21.15.43:44568) 2023-06-28T20:08:35.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.335+0000 7f2f45010700 1 -- 172.21.15.43:0/725240509 learned_addr learned my addr 172.21.15.43:0/725240509 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:35.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f4480f700 1 -- 172.21.15.43:0/725240509 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4047086147 0 0) 0x7f2f40070b40 con 0x7f2f401596a0 2023-06-28T20:08:35.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f45811700 1 --2- 172.21.15.43:0/725240509 >> v2:172.21.15.178:3300/0 conn(0x7f2f401592d0 0x7f2f400b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:35.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f4480f700 1 -- 172.21.15.43:0/725240509 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f2c0035b0 con 0x7f2f401596a0 2023-06-28T20:08:35.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f4480f700 1 -- 172.21.15.43:0/725240509 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 340797693 0 0) 0x7f2f2c0035b0 con 0x7f2f401596a0 2023-06-28T20:08:35.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f4480f700 1 -- 172.21.15.43:0/725240509 >> v2:172.21.15.178:3300/0 conn(0x7f2f401592d0 msgr2=0x7f2f400b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:35.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f4480f700 1 --2- 172.21.15.43:0/725240509 >> v2:172.21.15.178:3300/0 conn(0x7f2f401592d0 0x7f2f400b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f4480f700 1 -- 172.21.15.43:0/725240509 >> v1:172.21.15.178:6789/0 conn(0x7f2f40071900 legacy=0x7f2f400ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:35.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f4480f700 1 -- 172.21.15.43:0/725240509 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f400713e0 con 0x7f2f401596a0 2023-06-28T20:08:35.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f4480f700 1 -- 172.21.15.43:0/725240509 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f2f3c001670 con 0x7f2f401596a0 2023-06-28T20:08:35.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f4480f700 1 -- 172.21.15.43:0/725240509 >> v1:172.21.15.43:6789/0 conn(0x7f2f401596a0 legacy=0x7f2f4015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:35.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f4480f700 1 --2- 172.21.15.43:0/725240509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f2c003a00 0x7f2f2c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:35.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f4480f700 1 --2- 172.21.15.43:0/725240509 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f2c004850 0x7f2f2c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:35.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f4480f700 1 --2- 172.21.15.43:0/725240509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f2c005200 0x7f2f2c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:35.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f4480f700 1 -- 172.21.15.43:0/725240509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f40071100 con 0x7f2f2c003a00 2023-06-28T20:08:35.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f4480f700 1 -- 172.21.15.43:0/725240509 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c00c080 con 0x7f2f2c005200 2023-06-28T20:08:35.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f4480f700 1 -- 172.21.15.43:0/725240509 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c00c240 con 0x7f2f2c004850 2023-06-28T20:08:35.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f45811700 1 --2- 172.21.15.43:0/725240509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f2c003a00 0x7f2f2c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:35.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f46012700 1 --2- 172.21.15.43:0/725240509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f2c005200 0x7f2f2c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:35.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f45010700 1 --2- 172.21.15.43:0/725240509 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f2c004850 0x7f2f2c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:35.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f45811700 1 -- 172.21.15.43:0/725240509 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f2c004850 msgr2=0x7f2f2c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:35.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f45811700 1 --2- 172.21.15.43:0/725240509 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f2c004850 0x7f2f2c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f45811700 1 -- 172.21.15.43:0/725240509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f2c005200 msgr2=0x7f2f2c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:35.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f45811700 1 --2- 172.21.15.43:0/725240509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f2c005200 0x7f2f2c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f46012700 1 -- 172.21.15.43:0/725240509 reap_dead start 2023-06-28T20:08:35.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f45811700 1 -- 172.21.15.43:0/725240509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c0035b0 con 0x7f2f2c003a00 2023-06-28T20:08:35.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.336+0000 7f2f45010700 1 --2- 172.21.15.43:0/725240509 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f2c004850 0x7f2f2c004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:35.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.337+0000 7f2f45811700 1 --2- 172.21.15.43:0/725240509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f2c003a00 0x7f2f2c003e10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009b60 tx=0x7f2f2c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8d263fc564074 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:35.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.337+0000 7f2f4480f700 1 -- 172.21.15.43:0/725240509 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3000e6c0 con 0x7f2f2c003a00 2023-06-28T20:08:35.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.337+0000 7f2f4480f700 1 -- 172.21.15.43:0/725240509 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3000e820 con 0x7f2f2c003a00 2023-06-28T20:08:35.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.337+0000 7f2f4480f700 1 -- 172.21.15.43:0/725240509 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3000eb90 con 0x7f2f2c003a00 2023-06-28T20:08:35.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.337+0000 7f2f47a75700 1 -- 172.21.15.43:0/725240509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f2c003a00 msgr2=0x7f2f2c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:35.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.337+0000 7f2f47a75700 1 --2- 172.21.15.43:0/725240509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f2c003a00 0x7f2f2c003e10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009b60 tx=0x7f2f2c005880 comp rx=0 tx=0).stop 2023-06-28T20:08:35.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.337+0000 7f2f47a75700 1 -- 172.21.15.43:0/725240509 shutdown_connections 2023-06-28T20:08:35.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.337+0000 7f2f47a75700 1 --2- 172.21.15.43:0/725240509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f2c003a00 0x7f2f2c003e10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.337+0000 7f2f47a75700 1 -- 172.21.15.43:0/725240509 >> 172.21.15.43:0/725240509 conn(0x7f2f4006b000 msgr2=0x7f2f4006b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:35.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.337+0000 7f2f47a75700 1 -- 172.21.15.43:0/725240509 shutdown_connections 2023-06-28T20:08:35.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.337+0000 7f2f47a75700 1 -- 172.21.15.43:0/725240509 wait complete. 2023-06-28T20:08:35.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.339+0000 7f2f47a75700 1 Processor -- start 2023-06-28T20:08:35.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.339+0000 7f2f47a75700 1 -- start start 2023-06-28T20:08:35.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.340+0000 7f2f47a75700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f2c003a00 0x7f2f40151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:35.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.340+0000 7f2f47a75700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f40148c10 0x7f2f401482a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:35.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.340+0000 7f2f47a75700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f4015a740 0x7f2f4015abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:35.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.340+0000 7f2f47a75700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f40151c90 con 0x7f2f2c003a00 2023-06-28T20:08:35.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.340+0000 7f2f47a75700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f401506b0 con 0x7f2f40148c10 2023-06-28T20:08:35.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.340+0000 7f2f47a75700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f40150990 con 0x7f2f4015a740 2023-06-28T20:08:35.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.340+0000 7f2f45811700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f2c003a00 0x7f2f40151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:35.353 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.340+0000 7f2f45811700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f2c003a00 0x7f2f40151750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55392/0 (socket says 172.21.15.43:55392) 2023-06-28T20:08:35.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.340+0000 7f2f45811700 1 -- 172.21.15.43:0/4114109822 learned_addr learned my addr 172.21.15.43:0/4114109822 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:35.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.340+0000 7f2f45811700 1 -- 172.21.15.43:0/4114109822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f4015a740 msgr2=0x7f2f4015abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:35.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.340+0000 7f2f45811700 1 --2- 172.21.15.43:0/4114109822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f4015a740 0x7f2f4015abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.340+0000 7f2f45811700 1 -- 172.21.15.43:0/4114109822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f40148c10 msgr2=0x7f2f401482a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:35.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.340+0000 7f2f45811700 1 --2- 172.21.15.43:0/4114109822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f40148c10 0x7f2f401482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.340+0000 7f2f45811700 1 -- 172.21.15.43:0/4114109822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f40150c70 con 0x7f2f2c003a00 2023-06-28T20:08:35.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.341+0000 7f2f45811700 1 --2- 172.21.15.43:0/4114109822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f2c003a00 0x7f2f40151750 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00c050 tx=0x7f2f300047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:35.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.341+0000 7f2f367fc700 1 -- 172.21.15.43:0/4114109822 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3000e6c0 con 0x7f2f2c003a00 2023-06-28T20:08:35.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.341+0000 7f2f367fc700 1 -- 172.21.15.43:0/4114109822 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f30003ad0 con 0x7f2f2c003a00 2023-06-28T20:08:35.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.341+0000 7f2f47a75700 1 -- 172.21.15.43:0/4114109822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f40150f50 con 0x7f2f2c003a00 2023-06-28T20:08:35.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.341+0000 7f2f367fc700 1 -- 172.21.15.43:0/4114109822 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f300045d0 con 0x7f2f2c003a00 2023-06-28T20:08:35.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.341+0000 7f2f47a75700 1 -- 172.21.15.43:0/4114109822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f401487e0 con 0x7f2f2c003a00 2023-06-28T20:08:35.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.342+0000 7f2f47a75700 1 -- 172.21.15.43:0/4114109822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f401515a0 con 0x7f2f2c003a00 2023-06-28T20:08:35.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.342+0000 7f2f367fc700 1 -- 172.21.15.43:0/4114109822 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2f30013470 con 0x7f2f2c003a00 2023-06-28T20:08:35.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.342+0000 7f2f367fc700 1 --2- 172.21.15.43:0/4114109822 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2f2406ace0 0x7f2f2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:35.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.343+0000 7f2f367fc700 1 -- 172.21.15.43:0/4114109822 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2f3008ac30 con 0x7f2f2c003a00 2023-06-28T20:08:35.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.343+0000 7f2f45010700 1 --2- 172.21.15.43:0/4114109822 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2f2406ace0 0x7f2f2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:35.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.343+0000 7f2f45010700 1 --2- 172.21.15.43:0/4114109822 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2f2406ace0 0x7f2f2406d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c001bb0 tx=0x7f2f3c001150 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:35.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.347+0000 7f2f367fc700 1 -- 172.21.15.43:0/4114109822 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2f30055e90 con 0x7f2f2c003a00 2023-06-28T20:08:35.549 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.545+0000 7f2f47a75700 1 -- 172.21.15.43:0/4114109822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2f400cc3e0 con 0x7f2f2c003a00 2023-06-28T20:08:35.550 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.545+0000 7f2f367fc700 1 -- 172.21.15.43:0/4114109822 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v35) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2f300170a0 con 0x7f2f2c003a00 2023-06-28T20:08:35.551 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 8 2023-06-28T20:08:35.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.547+0000 7f2f47a75700 1 -- 172.21.15.43:0/4114109822 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2f2406ace0 msgr2=0x7f2f2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:35.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.547+0000 7f2f47a75700 1 --2- 172.21.15.43:0/4114109822 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2f2406ace0 0x7f2f2406d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c001bb0 tx=0x7f2f3c001150 comp rx=0 tx=0).stop 2023-06-28T20:08:35.553 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.547+0000 7f2f47a75700 1 -- 172.21.15.43:0/4114109822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f2c003a00 msgr2=0x7f2f40151750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:35.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.547+0000 7f2f47a75700 1 --2- 172.21.15.43:0/4114109822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f2c003a00 0x7f2f40151750 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00c050 tx=0x7f2f300047c0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.547+0000 7f2f47a75700 1 -- 172.21.15.43:0/4114109822 shutdown_connections 2023-06-28T20:08:35.554 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.547+0000 7f2f47a75700 1 --2- 172.21.15.43:0/4114109822 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f2f2406ace0 0x7f2f2406d190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.547+0000 7f2f47a75700 1 --2- 172.21.15.43:0/4114109822 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f2f4015a740 0x7f2f4015abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.555 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.547+0000 7f2f47a75700 1 --2- 172.21.15.43:0/4114109822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f2c003a00 0x7f2f40151750 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.556 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.547+0000 7f2f47a75700 1 --2- 172.21.15.43:0/4114109822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f40148c10 0x7f2f401482a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.547+0000 7f2f47a75700 1 -- 172.21.15.43:0/4114109822 >> 172.21.15.43:0/4114109822 conn(0x7f2f4006b000 msgr2=0x7f2f40154180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:35.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.548+0000 7f2f47a75700 1 -- 172.21.15.43:0/4114109822 shutdown_connections 2023-06-28T20:08:35.557 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.548+0000 7f2f47a75700 1 -- 172.21.15.43:0/4114109822 wait complete. 2023-06-28T20:08:35.564 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-28T20:08:35.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.690+0000 7fe6c2b7b700 1 Processor -- start 2023-06-28T20:08:35.694 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.690+0000 7fe6c2b7b700 1 -- start start 2023-06-28T20:08:35.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.691+0000 7fe6c2b7b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe6bc06d310 0x7fe6bc06cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:35.696 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.691+0000 7fe6c2b7b700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe6bc15cae0 0x7fe6bc06d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:35.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.691+0000 7fe6c2b7b700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6bc0d0680 con 0x7fe6bc0cfe00 2023-06-28T20:08:35.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.691+0000 7fe6c2b7b700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6bc06dc20 con 0x7fe6bc06d310 2023-06-28T20:08:35.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.691+0000 7fe6c2b7b700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6bc06bb20 con 0x7fe6bc15cae0 2023-06-28T20:08:35.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.691+0000 7fe6c0917700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe6bc06d310 0x7fe6bc06cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:35.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.691+0000 7fe6c0917700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe6bc06d310 0x7fe6bc06cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55400/0 (socket says 172.21.15.43:55400) 2023-06-28T20:08:35.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.691+0000 7fe6c0917700 1 -- 172.21.15.43:0/318833476 learned_addr learned my addr 172.21.15.43:0/318833476 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:35.697 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.691+0000 7fe6bbfff700 1 --2- 172.21.15.43:0/318833476 >> v2:172.21.15.178:3300/0 conn(0x7fe6bc15cae0 0x7fe6bc06d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:35.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.691+0000 7fe6c0917700 1 -- 172.21.15.43:0/318833476 >> v2:172.21.15.178:3300/0 conn(0x7fe6bc15cae0 msgr2=0x7fe6bc06d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:35.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.691+0000 7fe6c0917700 1 --2- 172.21.15.43:0/318833476 >> v2:172.21.15.178:3300/0 conn(0x7fe6bc15cae0 0x7fe6bc06d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.691+0000 7fe6c0917700 1 -- 172.21.15.43:0/318833476 >> v1:172.21.15.178:6789/0 conn(0x7fe6bc0cfe00 legacy=0x7fe6bc0ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:35.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.691+0000 7fe6c0917700 1 -- 172.21.15.43:0/318833476 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc06be00 con 0x7fe6bc06d310 2023-06-28T20:08:35.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.692+0000 7fe6c0917700 1 --2- 172.21.15.43:0/318833476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6bc06d310 0x7fe6bc06cbc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc0b37c0 tx=0x7fe6b0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a03ece6875ed7a7 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:35.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.692+0000 7fe6bb7fe700 1 -- 172.21.15.43:0/318833476 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0005e30 con 0x7fe6bc06d310 2023-06-28T20:08:35.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.692+0000 7fe6bb7fe700 1 -- 172.21.15.43:0/318833476 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b0006030 con 0x7fe6bc06d310 2023-06-28T20:08:35.698 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.692+0000 7fe6bb7fe700 1 -- 172.21.15.43:0/318833476 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b00063e0 con 0x7fe6bc06d310 2023-06-28T20:08:35.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.692+0000 7fe6c2b7b700 1 -- 172.21.15.43:0/318833476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6bc06d310 msgr2=0x7fe6bc06cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:35.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.692+0000 7fe6c2b7b700 1 --2- 172.21.15.43:0/318833476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6bc06d310 0x7fe6bc06cbc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc0b37c0 tx=0x7fe6b0009bd0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.692+0000 7fe6c2b7b700 1 -- 172.21.15.43:0/318833476 shutdown_connections 2023-06-28T20:08:35.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.692+0000 7fe6c2b7b700 1 --2- 172.21.15.43:0/318833476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6bc06d310 0x7fe6bc06cbc0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.692+0000 7fe6c2b7b700 1 --2- 172.21.15.43:0/318833476 >> v2:172.21.15.178:3300/0 conn(0x7fe6bc15cae0 0x7fe6bc06d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.692+0000 7fe6c2b7b700 1 -- 172.21.15.43:0/318833476 >> 172.21.15.43:0/318833476 conn(0x7fe6bc0664b0 msgr2=0x7fe6bc0d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:35.699 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.692+0000 7fe6c2b7b700 1 -- 172.21.15.43:0/318833476 shutdown_connections 2023-06-28T20:08:35.700 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.692+0000 7fe6c2b7b700 1 -- 172.21.15.43:0/318833476 wait complete. 2023-06-28T20:08:35.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.694+0000 7fe6c2b7b700 1 Processor -- start 2023-06-28T20:08:35.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.694+0000 7fe6c2b7b700 1 -- start start 2023-06-28T20:08:35.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.695+0000 7fe6c2b7b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6bc06d310 0x7fe6bc157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:35.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.695+0000 7fe6c2b7b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6bc0cfe00 0x7fe6bc0c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:35.702 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.695+0000 7fe6c2b7b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6bc15cae0 0x7fe6bc1573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:35.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.695+0000 7fe6c2b7b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc158520 con 0x7fe6bc15cae0 2023-06-28T20:08:35.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.695+0000 7fe6c2b7b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc0c97b0 con 0x7fe6bc06d310 2023-06-28T20:08:35.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.695+0000 7fe6c2b7b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc0c94d0 con 0x7fe6bc0cfe00 2023-06-28T20:08:35.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.695+0000 7fe6c1118700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6bc15cae0 0x7fe6bc1573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:35.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.695+0000 7fe6c1118700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6bc15cae0 0x7fe6bc1573b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55410/0 (socket says 172.21.15.43:55410) 2023-06-28T20:08:35.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.695+0000 7fe6c1118700 1 -- 172.21.15.43:0/3969262221 learned_addr learned my addr 172.21.15.43:0/3969262221 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:35.703 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.695+0000 7fe6bbfff700 1 --2- 172.21.15.43:0/3969262221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6bc0cfe00 0x7fe6bc0c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.695+0000 7fe6c0917700 1 --2- 172.21.15.43:0/3969262221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6bc06d310 0x7fe6bc157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.696+0000 7fe6c1118700 1 -- 172.21.15.43:0/3969262221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6bc0cfe00 msgr2=0x7fe6bc0c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.696+0000 7fe6c1118700 1 --2- 172.21.15.43:0/3969262221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6bc0cfe00 0x7fe6bc0c8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.696+0000 7fe6c1118700 1 -- 172.21.15.43:0/3969262221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6bc06d310 msgr2=0x7fe6bc157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.696+0000 7fe6c1118700 1 --2- 172.21.15.43:0/3969262221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6bc06d310 0x7fe6bc157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.696+0000 7fe6c1118700 1 -- 172.21.15.43:0/3969262221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b0009500 con 0x7fe6bc15cae0 2023-06-28T20:08:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.696+0000 7fe6c0917700 1 --2- 172.21.15.43:0/3969262221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6bc06d310 0x7fe6bc157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:35.704 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.696+0000 7fe6c1118700 1 --2- 172.21.15.43:0/3969262221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6bc15cae0 0x7fe6bc1573b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400cc10 tx=0x7fe6b400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:35.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.696+0000 7fe6b97fa700 1 -- 172.21.15.43:0/3969262221 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b400abb0 con 0x7fe6bc15cae0 2023-06-28T20:08:35.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.696+0000 7fe6b97fa700 1 -- 172.21.15.43:0/3969262221 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b400ad10 con 0x7fe6bc15cae0 2023-06-28T20:08:35.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.696+0000 7fe6c2b7b700 1 -- 172.21.15.43:0/3969262221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6bc0b1470 con 0x7fe6bc15cae0 2023-06-28T20:08:35.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.696+0000 7fe6b97fa700 1 -- 172.21.15.43:0/3969262221 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4012660 con 0x7fe6bc15cae0 2023-06-28T20:08:35.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.696+0000 7fe6c2b7b700 1 -- 172.21.15.43:0/3969262221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6bc0b1a30 con 0x7fe6bc15cae0 2023-06-28T20:08:35.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.697+0000 7fe6c2b7b700 1 -- 172.21.15.43:0/3969262221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6bc0728b0 con 0x7fe6bc15cae0 2023-06-28T20:08:35.705 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.697+0000 7fe6b97fa700 1 -- 172.21.15.43:0/3969262221 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe6b40127c0 con 0x7fe6bc15cae0 2023-06-28T20:08:35.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.698+0000 7fe6b97fa700 1 --2- 172.21.15.43:0/3969262221 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe6a006ace0 0x7fe6a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:35.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.698+0000 7fe6b97fa700 1 -- 172.21.15.43:0/3969262221 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe6b4089c00 con 0x7fe6bc15cae0 2023-06-28T20:08:35.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.698+0000 7fe6c0917700 1 --2- 172.21.15.43:0/3969262221 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe6a006ace0 0x7fe6a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:35.708 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.698+0000 7fe6c0917700 1 --2- 172.21.15.43:0/3969262221 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe6a006ace0 0x7fe6a006d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc0b37c0 tx=0x7fe6b0008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:35.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.702+0000 7fe6b97fa700 1 -- 172.21.15.43:0/3969262221 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe6b4090080 con 0x7fe6bc15cae0 2023-06-28T20:08:35.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.900+0000 7fe6c2b7b700 1 -- 172.21.15.43:0/3969262221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe6bc06a2a0 con 0x7fe6bc15cae0 2023-06-28T20:08:35.904 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.900+0000 7fe6b97fa700 1 -- 172.21.15.43:0/3969262221 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe6b4016020 con 0x7fe6bc15cae0 2023-06-28T20:08:35.905 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2023-06-28T20:08:35.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.902+0000 7fe6c2b7b700 1 -- 172.21.15.43:0/3969262221 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe6a006ace0 msgr2=0x7fe6a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:35.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.902+0000 7fe6c2b7b700 1 --2- 172.21.15.43:0/3969262221 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe6a006ace0 0x7fe6a006d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc0b37c0 tx=0x7fe6b0008070 comp rx=0 tx=0).stop 2023-06-28T20:08:35.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.902+0000 7fe6c2b7b700 1 -- 172.21.15.43:0/3969262221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6bc15cae0 msgr2=0x7fe6bc1573b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:35.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.902+0000 7fe6c2b7b700 1 --2- 172.21.15.43:0/3969262221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6bc15cae0 0x7fe6bc1573b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400cc10 tx=0x7fe6b400a060 comp rx=0 tx=0).stop 2023-06-28T20:08:35.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.903+0000 7fe6c2b7b700 1 -- 172.21.15.43:0/3969262221 shutdown_connections 2023-06-28T20:08:35.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.903+0000 7fe6c2b7b700 1 --2- 172.21.15.43:0/3969262221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6bc15cae0 0x7fe6bc1573b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.903+0000 7fe6c2b7b700 1 --2- 172.21.15.43:0/3969262221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6bc06d310 0x7fe6bc157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.903+0000 7fe6c2b7b700 1 --2- 172.21.15.43:0/3969262221 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe6a006ace0 0x7fe6a006d190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.903+0000 7fe6c2b7b700 1 --2- 172.21.15.43:0/3969262221 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6bc0cfe00 0x7fe6bc0c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:35.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.903+0000 7fe6c2b7b700 1 -- 172.21.15.43:0/3969262221 >> 172.21.15.43:0/3969262221 conn(0x7fe6bc0664b0 msgr2=0x7fe6bc06ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:35.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.903+0000 7fe6c2b7b700 1 -- 172.21.15.43:0/3969262221 shutdown_connections 2023-06-28T20:08:35.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:35.903+0000 7fe6c2b7b700 1 -- 172.21.15.43:0/3969262221 wait complete. 2023-06-28T20:08:35.919 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-28T20:08:36.047 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.043+0000 7f5d990bd700 1 Processor -- start 2023-06-28T20:08:36.048 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.043+0000 7f5d990bd700 1 -- start start 2023-06-28T20:08:36.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.044+0000 7f5d990bd700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d9415cf10 0x7f5d940b82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:36.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.044+0000 7f5d990bd700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d940d0f00 con 0x7f5d940d0a80 2023-06-28T20:08:36.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.044+0000 7f5d990bd700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d940b8c80 con 0x7f5d940cf830 2023-06-28T20:08:36.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.044+0000 7f5d990bd700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d940d11a0 con 0x7f5d9415cf10 2023-06-28T20:08:36.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.044+0000 7f5d9259c700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5d940d0a80 0x7f5d940d1c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44602/0 (socket says 172.21.15.43:44602) 2023-06-28T20:08:36.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.044+0000 7f5d9259c700 1 -- 172.21.15.43:0/3814196832 learned_addr learned my addr 172.21.15.43:0/3814196832 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:36.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.044+0000 7f5d9359e700 1 --2- 172.21.15.43:0/3814196832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d9415cf10 0x7f5d940b82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:36.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.044+0000 7f5d91d9b700 1 -- 172.21.15.43:0/3814196832 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028931883 0 0) 0x7f5d940d0f00 con 0x7f5d940d0a80 2023-06-28T20:08:36.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.045+0000 7f5d91d9b700 1 -- 172.21.15.43:0/3814196832 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d7c0035b0 con 0x7f5d940d0a80 2023-06-28T20:08:36.052 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.045+0000 7f5d91d9b700 1 -- 172.21.15.43:0/3814196832 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 362078213 0 0) 0x7f5d940b8c80 con 0x7f5d940cf830 2023-06-28T20:08:36.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.045+0000 7f5d91d9b700 1 -- 172.21.15.43:0/3814196832 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d940d0f00 con 0x7f5d940cf830 2023-06-28T20:08:36.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.045+0000 7f5d9359e700 1 -- 172.21.15.43:0/3814196832 >> v1:172.21.15.178:6789/0 conn(0x7f5d940cf830 legacy=0x7f5d940cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:36.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.045+0000 7f5d91d9b700 1 -- 172.21.15.43:0/3814196832 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2672211663 0 0) 0x7f5d7c0035b0 con 0x7f5d940d0a80 2023-06-28T20:08:36.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.045+0000 7f5d9359e700 1 -- 172.21.15.43:0/3814196832 >> v1:172.21.15.43:6789/0 conn(0x7f5d940d0a80 legacy=0x7f5d940d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:36.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.045+0000 7f5d9359e700 1 -- 172.21.15.43:0/3814196832 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d940d1480 con 0x7f5d9415cf10 2023-06-28T20:08:36.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.045+0000 7f5d9359e700 1 --2- 172.21.15.43:0/3814196832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d9415cf10 0x7f5d940b82e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009dc0 tx=0x7f5d8c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67bb0352aec569f server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:36.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.045+0000 7f5d91d9b700 1 -- 172.21.15.43:0/3814196832 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c00e070 con 0x7f5d9415cf10 2023-06-28T20:08:36.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.045+0000 7f5d91d9b700 1 -- 172.21.15.43:0/3814196832 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8c003a20 con 0x7f5d9415cf10 2023-06-28T20:08:36.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.045+0000 7f5d91d9b700 1 -- 172.21.15.43:0/3814196832 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84001670 con 0x7f5d9415cf10 2023-06-28T20:08:36.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.046+0000 7f5d990bd700 1 -- 172.21.15.43:0/3814196832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d9415cf10 msgr2=0x7f5d940b82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:36.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.046+0000 7f5d990bd700 1 --2- 172.21.15.43:0/3814196832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d9415cf10 0x7f5d940b82e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c009dc0 tx=0x7f5d8c00b330 comp rx=0 tx=0).stop 2023-06-28T20:08:36.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.046+0000 7f5d990bd700 1 -- 172.21.15.43:0/3814196832 shutdown_connections 2023-06-28T20:08:36.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.046+0000 7f5d990bd700 1 --2- 172.21.15.43:0/3814196832 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d9415cf10 0x7f5d940b82e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:36.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.046+0000 7f5d990bd700 1 -- 172.21.15.43:0/3814196832 >> 172.21.15.43:0/3814196832 conn(0x7f5d9406a640 msgr2=0x7f5d9406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:36.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.046+0000 7f5d990bd700 1 -- 172.21.15.43:0/3814196832 shutdown_connections 2023-06-28T20:08:36.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.046+0000 7f5d990bd700 1 -- 172.21.15.43:0/3814196832 wait complete. 2023-06-28T20:08:36.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.047+0000 7f5d990bd700 1 Processor -- start 2023-06-28T20:08:36.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.047+0000 7f5d990bd700 1 -- start start 2023-06-28T20:08:36.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.048+0000 7f5d990bd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d940cf830 0x7f5d94159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:36.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.048+0000 7f5d990bd700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d940d0a80 0x7f5d94152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:36.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.048+0000 7f5d990bd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d9415cf10 0x7f5d940cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:36.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.048+0000 7f5d990bd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d94159cb0 con 0x7f5d9415cf10 2023-06-28T20:08:36.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.048+0000 7f5d990bd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d941568f0 con 0x7f5d940cf830 2023-06-28T20:08:36.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.048+0000 7f5d990bd700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d94153180 con 0x7f5d940d0a80 2023-06-28T20:08:36.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.048+0000 7f5d9359e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d9415cf10 0x7f5d940cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:36.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.048+0000 7f5d9359e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d9415cf10 0x7f5d940cb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55424/0 (socket says 172.21.15.43:55424) 2023-06-28T20:08:36.059 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.048+0000 7f5d9359e700 1 -- 172.21.15.43:0/4062650499 learned_addr learned my addr 172.21.15.43:0/4062650499 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:36.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.048+0000 7f5d9259c700 1 --2- 172.21.15.43:0/4062650499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d940d0a80 0x7f5d94152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:36.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.048+0000 7f5d92d9d700 1 --2- 172.21.15.43:0/4062650499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d940cf830 0x7f5d94159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:36.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.048+0000 7f5d9359e700 1 -- 172.21.15.43:0/4062650499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d940d0a80 msgr2=0x7f5d94152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:36.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.048+0000 7f5d9359e700 1 --2- 172.21.15.43:0/4062650499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d940d0a80 0x7f5d94152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:36.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.049+0000 7f5d9359e700 1 -- 172.21.15.43:0/4062650499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d940cf830 msgr2=0x7f5d94159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:36.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.049+0000 7f5d9359e700 1 --2- 172.21.15.43:0/4062650499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d940cf830 0x7f5d94159770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:36.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.049+0000 7f5d9359e700 1 -- 172.21.15.43:0/4062650499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c009a70 con 0x7f5d9415cf10 2023-06-28T20:08:36.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.049+0000 7f5d92d9d700 1 --2- 172.21.15.43:0/4062650499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d940cf830 0x7f5d94159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:36.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.049+0000 7f5d9359e700 1 --2- 172.21.15.43:0/4062650499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d9415cf10 0x7f5d940cb4e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5d940b5a10 tx=0x7f5d8c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:36.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.049+0000 7f5d7b7fe700 1 -- 172.21.15.43:0/4062650499 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c004540 con 0x7f5d9415cf10 2023-06-28T20:08:36.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.049+0000 7f5d990bd700 1 -- 172.21.15.43:0/4062650499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d940b2ed0 con 0x7f5d9415cf10 2023-06-28T20:08:36.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.049+0000 7f5d7b7fe700 1 -- 172.21.15.43:0/4062650499 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8c0046a0 con 0x7f5d9415cf10 2023-06-28T20:08:36.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.049+0000 7f5d7b7fe700 1 -- 172.21.15.43:0/4062650499 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c01d470 con 0x7f5d9415cf10 2023-06-28T20:08:36.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.049+0000 7f5d990bd700 1 -- 172.21.15.43:0/4062650499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d940b3490 con 0x7f5d9415cf10 2023-06-28T20:08:36.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.050+0000 7f5d990bd700 1 -- 172.21.15.43:0/4062650499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d940728b0 con 0x7f5d9415cf10 2023-06-28T20:08:36.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.050+0000 7f5d7b7fe700 1 -- 172.21.15.43:0/4062650499 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5d8c01d5d0 con 0x7f5d9415cf10 2023-06-28T20:08:36.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.051+0000 7f5d7b7fe700 1 --2- 172.21.15.43:0/4062650499 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5d8006ace0 0x7f5d8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:36.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.051+0000 7f5d7b7fe700 1 -- 172.21.15.43:0/4062650499 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5d8c08aed0 con 0x7f5d9415cf10 2023-06-28T20:08:36.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.051+0000 7f5d92d9d700 1 --2- 172.21.15.43:0/4062650499 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5d8006ace0 0x7f5d8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:36.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.051+0000 7f5d92d9d700 1 --2- 172.21.15.43:0/4062650499 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5d8006ace0 0x7f5d8006d190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5d88007fd0 tx=0x7f5d88009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:36.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.055+0000 7f5d7b7fe700 1 -- 172.21.15.43:0/4062650499 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5d8c055f90 con 0x7f5d9415cf10 2023-06-28T20:08:36.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.252+0000 7f5d990bd700 1 -- 172.21.15.43:0/4062650499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5d9406aa10 con 0x7f5d9415cf10 2023-06-28T20:08:36.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.252+0000 7f5d7b7fe700 1 -- 172.21.15.43:0/4062650499 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5d8c0595b0 con 0x7f5d9415cf10 2023-06-28T20:08:36.257 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2023-06-28T20:08:36.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.254+0000 7f5d990bd700 1 -- 172.21.15.43:0/4062650499 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5d8006ace0 msgr2=0x7f5d8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:36.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.254+0000 7f5d990bd700 1 --2- 172.21.15.43:0/4062650499 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5d8006ace0 0x7f5d8006d190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5d88007fd0 tx=0x7f5d88009040 comp rx=0 tx=0).stop 2023-06-28T20:08:36.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.254+0000 7f5d990bd700 1 -- 172.21.15.43:0/4062650499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d9415cf10 msgr2=0x7f5d940cb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:36.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.254+0000 7f5d990bd700 1 --2- 172.21.15.43:0/4062650499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d9415cf10 0x7f5d940cb4e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5d940b5a10 tx=0x7f5d8c004ad0 comp rx=0 tx=0).stop 2023-06-28T20:08:36.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.254+0000 7f5d990bd700 1 -- 172.21.15.43:0/4062650499 shutdown_connections 2023-06-28T20:08:36.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.254+0000 7f5d990bd700 1 --2- 172.21.15.43:0/4062650499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d9415cf10 0x7f5d940cb4e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:36.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.254+0000 7f5d990bd700 1 --2- 172.21.15.43:0/4062650499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d940cf830 0x7f5d94159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.254+0000 7f5d990bd700 1 --2- 172.21.15.43:0/4062650499 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5d8006ace0 0x7f5d8006d190 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.254+0000 7f5d990bd700 1 --2- 172.21.15.43:0/4062650499 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5d940d0a80 0x7f5d94152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.254+0000 7f5d990bd700 1 -- 172.21.15.43:0/4062650499 >> 172.21.15.43:0/4062650499 conn(0x7f5d9406a640 msgr2=0x7f5d9415f340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.254+0000 7f5d990bd700 1 -- 172.21.15.43:0/4062650499 shutdown_connections 2023-06-28T20:08:36.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.254+0000 7f5d990bd700 1 -- 172.21.15.43:0/4062650499 wait complete. 2023-06-28T20:08:36.271 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-28T20:08:36.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.396+0000 7f1648f62700 1 Processor -- start 2023-06-28T20:08:36.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.397+0000 7f1648f62700 1 -- start start 2023-06-28T20:08:36.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.397+0000 7f1648f62700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f16440afdf0 0x7f164406da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:36.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.397+0000 7f1648f62700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16440ae950 con 0x7f16440ceee0 2023-06-28T20:08:36.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.397+0000 7f1648f62700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164415a9c0 con 0x7f164415a5f0 2023-06-28T20:08:36.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.397+0000 7f1648f62700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16440add40 con 0x7f16440afdf0 2023-06-28T20:08:36.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.398+0000 7f1641d9b700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f16440ceee0 0x7f164406d310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44614/0 (socket says 172.21.15.43:44614) 2023-06-28T20:08:36.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.398+0000 7f1641d9b700 1 -- 172.21.15.43:0/654495403 learned_addr learned my addr 172.21.15.43:0/654495403 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:36.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.398+0000 7f1642d9d700 1 --2- 172.21.15.43:0/654495403 >> v2:172.21.15.178:3300/0 conn(0x7f16440afdf0 0x7f164406da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:36.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.398+0000 7f164159a700 1 -- 172.21.15.43:0/654495403 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 929449495 0 0) 0x7f16440ae950 con 0x7f16440ceee0 2023-06-28T20:08:36.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.398+0000 7f164159a700 1 -- 172.21.15.43:0/654495403 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16300035b0 con 0x7f16440ceee0 2023-06-28T20:08:36.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.398+0000 7f164159a700 1 -- 172.21.15.43:0/654495403 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3633015578 0 0) 0x7f164415a9c0 con 0x7f164415a5f0 2023-06-28T20:08:36.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.398+0000 7f164159a700 1 -- 172.21.15.43:0/654495403 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16440ae950 con 0x7f164415a5f0 2023-06-28T20:08:36.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.398+0000 7f164159a700 1 -- 172.21.15.43:0/654495403 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3659268379 0 0) 0x7f16300035b0 con 0x7f16440ceee0 2023-06-28T20:08:36.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.398+0000 7f164159a700 1 -- 172.21.15.43:0/654495403 >> v2:172.21.15.178:3300/0 conn(0x7f16440afdf0 msgr2=0x7f164406da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:36.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.398+0000 7f164159a700 1 --2- 172.21.15.43:0/654495403 >> v2:172.21.15.178:3300/0 conn(0x7f16440afdf0 0x7f164406da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:36.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.398+0000 7f164159a700 1 -- 172.21.15.43:0/654495403 >> v1:172.21.15.178:6789/0 conn(0x7f164415a5f0 legacy=0x7f16440ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:36.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.398+0000 7f164159a700 1 -- 172.21.15.43:0/654495403 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16440ae020 con 0x7f16440ceee0 2023-06-28T20:08:36.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.398+0000 7f164159a700 1 -- 172.21.15.43:0/654495403 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f162c001670 con 0x7f16440ceee0 2023-06-28T20:08:36.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.398+0000 7f1642d9d700 1 --2- 172.21.15.43:0/654495403 >> v2:172.21.15.178:3300/0 conn(0x7f16440afdf0 0x7f164406da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:08:36.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.398+0000 7f164159a700 1 -- 172.21.15.43:0/654495403 >> v1:172.21.15.43:6789/0 conn(0x7f16440ceee0 legacy=0x7f164406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:36.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.398+0000 7f164159a700 1 --2- 172.21.15.43:0/654495403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1630003be0 0x7f1630003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:36.414 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.398+0000 7f164159a700 1 --2- 172.21.15.43:0/654495403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1630004850 0x7f1630004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:36.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.398+0000 7f164159a700 1 --2- 172.21.15.43:0/654495403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1630005200 0x7f163000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:36.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.398+0000 7f164159a700 1 -- 172.21.15.43:0/654495403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16440add40 con 0x7f1630005200 2023-06-28T20:08:36.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.398+0000 7f164159a700 1 -- 172.21.15.43:0/654495403 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163000c0e0 con 0x7f1630003be0 2023-06-28T20:08:36.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.398+0000 7f164159a700 1 -- 172.21.15.43:0/654495403 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163000c270 con 0x7f1630004850 2023-06-28T20:08:36.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.398+0000 7f1642d9d700 1 --2- 172.21.15.43:0/654495403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1630005200 0x7f163000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:36.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.398+0000 7f164259c700 1 --2- 172.21.15.43:0/654495403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1630003be0 0x7f1630003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:36.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.399+0000 7f1641d9b700 1 --2- 172.21.15.43:0/654495403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1630004850 0x7f1630004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:36.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.399+0000 7f1642d9d700 1 -- 172.21.15.43:0/654495403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1630004850 msgr2=0x7f1630004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:36.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.399+0000 7f1642d9d700 1 --2- 172.21.15.43:0/654495403 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1630004850 0x7f1630004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:36.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.399+0000 7f1642d9d700 1 -- 172.21.15.43:0/654495403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1630003be0 msgr2=0x7f1630003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:36.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.399+0000 7f1642d9d700 1 --2- 172.21.15.43:0/654495403 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1630003be0 0x7f1630003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:36.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.399+0000 7f1642d9d700 1 -- 172.21.15.43:0/654495403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16300035b0 con 0x7f1630005200 2023-06-28T20:08:36.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.399+0000 7f1642d9d700 1 -- 172.21.15.43:0/654495403 reap_dead start 2023-06-28T20:08:36.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.399+0000 7f1642d9d700 1 --2- 172.21.15.43:0/654495403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1630005200 0x7f163000ba10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f163c00e320 tx=0x7f1630005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655e2fad7fa140e0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:36.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.399+0000 7f164159a700 1 -- 172.21.15.43:0/654495403 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c00edc0 con 0x7f1630005200 2023-06-28T20:08:36.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.399+0000 7f164159a700 1 -- 172.21.15.43:0/654495403 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163c00f040 con 0x7f1630005200 2023-06-28T20:08:36.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.399+0000 7f164159a700 1 -- 172.21.15.43:0/654495403 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c0146c0 con 0x7f1630005200 2023-06-28T20:08:36.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.400+0000 7f1648f62700 1 -- 172.21.15.43:0/654495403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1630005200 msgr2=0x7f163000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:36.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.400+0000 7f1648f62700 1 --2- 172.21.15.43:0/654495403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1630005200 0x7f163000ba10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f163c00e320 tx=0x7f1630005880 comp rx=0 tx=0).stop 2023-06-28T20:08:36.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.400+0000 7f1648f62700 1 -- 172.21.15.43:0/654495403 shutdown_connections 2023-06-28T20:08:36.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.400+0000 7f1648f62700 1 --2- 172.21.15.43:0/654495403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1630005200 0x7f163000ba10 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:36.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.400+0000 7f1648f62700 1 -- 172.21.15.43:0/654495403 >> 172.21.15.43:0/654495403 conn(0x7f16440664b0 msgr2=0x7f16440d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:36.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.400+0000 7f1648f62700 1 -- 172.21.15.43:0/654495403 shutdown_connections 2023-06-28T20:08:36.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.400+0000 7f1648f62700 1 -- 172.21.15.43:0/654495403 wait complete. 2023-06-28T20:08:36.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.401+0000 7f1648f62700 1 Processor -- start 2023-06-28T20:08:36.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.401+0000 7f1648f62700 1 -- start start 2023-06-28T20:08:36.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.402+0000 7f1648f62700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1630005200 0x7f16440c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:36.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.402+0000 7f1648f62700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1644154c90 0x7f16440cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:36.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.402+0000 7f1648f62700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16440ce020 0x7f16441e0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:36.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.402+0000 7f1648f62700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16440c6450 con 0x7f1644154c90 2023-06-28T20:08:36.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.402+0000 7f1648f62700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16441f1db0 con 0x7f16440ce020 2023-06-28T20:08:36.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.402+0000 7f1648f62700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f164406b1a0 con 0x7f1630005200 2023-06-28T20:08:36.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.402+0000 7f1641d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1644154c90 0x7f16440cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:36.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.402+0000 7f1641d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1644154c90 0x7f16440cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55440/0 (socket says 172.21.15.43:55440) 2023-06-28T20:08:36.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.402+0000 7f1641d9b700 1 -- 172.21.15.43:0/77554083 learned_addr learned my addr 172.21.15.43:0/77554083 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:36.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.402+0000 7f164259c700 1 --2- 172.21.15.43:0/77554083 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1630005200 0x7f16440c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:36.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.402+0000 7f1642d9d700 1 --2- 172.21.15.43:0/77554083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16440ce020 0x7f16441e0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:36.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.402+0000 7f1641d9b700 1 -- 172.21.15.43:0/77554083 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1630005200 msgr2=0x7f16440c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:36.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.402+0000 7f1641d9b700 1 --2- 172.21.15.43:0/77554083 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1630005200 0x7f16440c5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:36.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.402+0000 7f1641d9b700 1 -- 172.21.15.43:0/77554083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16440ce020 msgr2=0x7f16441e0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:36.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.402+0000 7f1641d9b700 1 --2- 172.21.15.43:0/77554083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16440ce020 0x7f16441e0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:36.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.402+0000 7f1641d9b700 1 -- 172.21.15.43:0/77554083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164406b480 con 0x7f1644154c90 2023-06-28T20:08:36.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.403+0000 7f1641d9b700 1 --2- 172.21.15.43:0/77554083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1644154c90 0x7f16440cdae0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f162c004f50 tx=0x7f162c007340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:36.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.403+0000 7f163affd700 1 -- 172.21.15.43:0/77554083 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c012560 con 0x7f1644154c90 2023-06-28T20:08:36.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.403+0000 7f163affd700 1 -- 172.21.15.43:0/77554083 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162c0126c0 con 0x7f1644154c90 2023-06-28T20:08:36.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.403+0000 7f1648f62700 1 -- 172.21.15.43:0/77554083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164406b760 con 0x7f1644154c90 2023-06-28T20:08:36.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.403+0000 7f163affd700 1 -- 172.21.15.43:0/77554083 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c012a50 con 0x7f1644154c90 2023-06-28T20:08:36.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.403+0000 7f1648f62700 1 -- 172.21.15.43:0/77554083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164406bd20 con 0x7f1644154c90 2023-06-28T20:08:36.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.404+0000 7f1648f62700 1 -- 172.21.15.43:0/77554083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f164406b2e0 con 0x7f1644154c90 2023-06-28T20:08:36.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.404+0000 7f163affd700 1 -- 172.21.15.43:0/77554083 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f162c012bb0 con 0x7f1644154c90 2023-06-28T20:08:36.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.405+0000 7f163affd700 1 --2- 172.21.15.43:0/77554083 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f162406ace0 0x7f162406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:36.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.405+0000 7f163affd700 1 -- 172.21.15.43:0/77554083 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f162c0893c0 con 0x7f1644154c90 2023-06-28T20:08:36.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.405+0000 7f164259c700 1 --2- 172.21.15.43:0/77554083 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f162406ace0 0x7f162406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:36.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.405+0000 7f164259c700 1 --2- 172.21.15.43:0/77554083 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f162406ace0 0x7f162406d190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1634000c00 tx=0x7f1634009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:36.425 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.409+0000 7f163affd700 1 -- 172.21.15.43:0/77554083 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f164406b2e0 con 0x7f1644154c90 2023-06-28T20:08:36.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.607+0000 7f1648f62700 1 -- 172.21.15.43:0/77554083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f164406b2e0 con 0x7f1644154c90 2023-06-28T20:08:36.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.607+0000 7f163affd700 1 -- 172.21.15.43:0/77554083 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7f164406b2e0 con 0x7f1644154c90 2023-06-28T20:08:36.612 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2023-06-28T20:08:36.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.610+0000 7f1648f62700 1 -- 172.21.15.43:0/77554083 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f162406ace0 msgr2=0x7f162406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:36.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.610+0000 7f1648f62700 1 --2- 172.21.15.43:0/77554083 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f162406ace0 0x7f162406d190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1634000c00 tx=0x7f1634009040 comp rx=0 tx=0).stop 2023-06-28T20:08:36.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.610+0000 7f1648f62700 1 -- 172.21.15.43:0/77554083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1644154c90 msgr2=0x7f16440cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:36.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.610+0000 7f1648f62700 1 --2- 172.21.15.43:0/77554083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1644154c90 0x7f16440cdae0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f162c004f50 tx=0x7f162c007340 comp rx=0 tx=0).stop 2023-06-28T20:08:36.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.610+0000 7f1648f62700 1 -- 172.21.15.43:0/77554083 shutdown_connections 2023-06-28T20:08:36.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.610+0000 7f1648f62700 1 --2- 172.21.15.43:0/77554083 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f16440ce020 0x7f16441e0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:36.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.610+0000 7f1648f62700 1 --2- 172.21.15.43:0/77554083 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f162406ace0 0x7f162406d190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:36.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.610+0000 7f1648f62700 1 --2- 172.21.15.43:0/77554083 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1630005200 0x7f16440c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:36.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.610+0000 7f1648f62700 1 --2- 172.21.15.43:0/77554083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1644154c90 0x7f16440cdae0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:36.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.610+0000 7f1648f62700 1 -- 172.21.15.43:0/77554083 >> 172.21.15.43:0/77554083 conn(0x7f16440664b0 msgr2=0x7f16440ce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:36.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.610+0000 7f1648f62700 1 -- 172.21.15.43:0/77554083 shutdown_connections 2023-06-28T20:08:36.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.610+0000 7f1648f62700 1 -- 172.21.15.43:0/77554083 wait complete. 2023-06-28T20:08:36.626 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-28T20:08:36.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.749+0000 7f9147a2a700 1 Processor -- start 2023-06-28T20:08:36.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.749+0000 7f9147a2a700 1 -- start start 2023-06-28T20:08:36.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.750+0000 7f9147a2a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91400af060 0x7f91400cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:36.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.750+0000 7f9147a2a700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f914006ca80 0x7f9140069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:36.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.750+0000 7f9147a2a700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9140071410 con 0x7f914006ce80 2023-06-28T20:08:36.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.750+0000 7f9147a2a700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91400716b0 con 0x7f914006ca80 2023-06-28T20:08:36.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.750+0000 7f9147a2a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9140071990 con 0x7f91400af060 2023-06-28T20:08:36.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.750+0000 7f9144fc5700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f914006ca80 0x7f9140069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:36.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.750+0000 7f91457c6700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91400af060 0x7f91400cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:36.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.751+0000 7f9144fc5700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f914006ca80 0x7f9140069ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:41512/0 (socket says 172.21.15.43:41512) 2023-06-28T20:08:36.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.751+0000 7f91457c6700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91400af060 0x7f91400cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3301/0 says I am v2:172.21.15.43:39420/0 (socket says 172.21.15.43:39420) 2023-06-28T20:08:36.757 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.751+0000 7f9144fc5700 1 -- 172.21.15.43:0/3286702878 learned_addr learned my addr 172.21.15.43:0/3286702878 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:36.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.751+0000 7f9145fc7700 1 --1- 172.21.15.43:0/3286702878 >> v1:172.21.15.178:6789/0 conn(0x7f914006ce80 0x7f91400ac870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:43760/0 (socket says 172.21.15.43:43760) 2023-06-28T20:08:36.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.751+0000 7f91457c6700 1 -- 172.21.15.43:0/3286702878 >> v2:172.21.15.178:3300/0 conn(0x7f914006ca80 msgr2=0x7f9140069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:36.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.751+0000 7f91457c6700 1 --2- 172.21.15.43:0/3286702878 >> v2:172.21.15.178:3300/0 conn(0x7f914006ca80 0x7f9140069ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:36.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.751+0000 7f91457c6700 1 -- 172.21.15.43:0/3286702878 >> v1:172.21.15.178:6789/0 conn(0x7f914006ce80 legacy=0x7f91400ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:36.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.751+0000 7f91457c6700 1 -- 172.21.15.43:0/3286702878 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91400af430 con 0x7f91400af060 2023-06-28T20:08:36.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.751+0000 7f9137fff700 1 -- 172.21.15.43:0/3286702878 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3430535375 0 0) 0x7f9140071410 con 0x7f914006ce80 2023-06-28T20:08:36.758 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.751+0000 7f9144fc5700 1 --2- 172.21.15.43:0/3286702878 >> v2:172.21.15.178:3300/0 conn(0x7f914006ca80 0x7f9140069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:08:36.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.751+0000 7f91457c6700 1 --2- 172.21.15.43:0/3286702878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91400af060 0x7f91400cf680 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9130009840 tx=0x7f9130009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=866e57ed52e606f1 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:36.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.751+0000 7f9137fff700 1 -- 172.21.15.43:0/3286702878 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9130011830 con 0x7f91400af060 2023-06-28T20:08:36.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.751+0000 7f9137fff700 1 -- 172.21.15.43:0/3286702878 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9130011990 con 0x7f91400af060 2023-06-28T20:08:36.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.751+0000 7f9137fff700 1 -- 172.21.15.43:0/3286702878 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9130011d20 con 0x7f91400af060 2023-06-28T20:08:36.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.752+0000 7f9147a2a700 1 -- 172.21.15.43:0/3286702878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91400af060 msgr2=0x7f91400cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:36.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.752+0000 7f9147a2a700 1 --2- 172.21.15.43:0/3286702878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91400af060 0x7f91400cf680 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9130009840 tx=0x7f9130009b50 comp rx=0 tx=0).stop 2023-06-28T20:08:36.759 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.752+0000 7f9147a2a700 1 -- 172.21.15.43:0/3286702878 shutdown_connections 2023-06-28T20:08:36.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.752+0000 7f9147a2a700 1 --2- 172.21.15.43:0/3286702878 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91400af060 0x7f91400cf680 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:36.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.752+0000 7f9147a2a700 1 --2- 172.21.15.43:0/3286702878 >> v2:172.21.15.178:3300/0 conn(0x7f914006ca80 0x7f9140069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:36.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.752+0000 7f9147a2a700 1 -- 172.21.15.43:0/3286702878 >> 172.21.15.43:0/3286702878 conn(0x7f91400b1720 msgr2=0x7f91400b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:36.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.752+0000 7f9147a2a700 1 -- 172.21.15.43:0/3286702878 shutdown_connections 2023-06-28T20:08:36.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.752+0000 7f9147a2a700 1 -- 172.21.15.43:0/3286702878 wait complete. 2023-06-28T20:08:36.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.754+0000 7f9147a2a700 1 Processor -- start 2023-06-28T20:08:36.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.754+0000 7f9147a2a700 1 -- start start 2023-06-28T20:08:36.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.755+0000 7f9147a2a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f914006ce80 0x7f91401f1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:36.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.755+0000 7f9147a2a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91400af060 0x7f91401f54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:36.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.755+0000 7f9147a2a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91401f5a00 0x7f91400ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:36.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.755+0000 7f9147a2a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91401f5ea0 con 0x7f914006ce80 2023-06-28T20:08:36.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.755+0000 7f9147a2a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914015e220 con 0x7f91400af060 2023-06-28T20:08:36.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.755+0000 7f9147a2a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914015d610 con 0x7f91401f5a00 2023-06-28T20:08:36.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.755+0000 7f91457c6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f914006ce80 0x7f91401f1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:36.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.755+0000 7f91457c6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f914006ce80 0x7f91401f1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55452/0 (socket says 172.21.15.43:55452) 2023-06-28T20:08:36.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.755+0000 7f91457c6700 1 -- 172.21.15.43:0/98119413 learned_addr learned my addr 172.21.15.43:0/98119413 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:36.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.755+0000 7f9145fc7700 1 --2- 172.21.15.43:0/98119413 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91401f5a00 0x7f91400ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:36.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.755+0000 7f9144fc5700 1 --2- 172.21.15.43:0/98119413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91400af060 0x7f91401f54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:36.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.755+0000 7f91457c6700 1 -- 172.21.15.43:0/98119413 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91401f5a00 msgr2=0x7f91400ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:36.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.755+0000 7f91457c6700 1 --2- 172.21.15.43:0/98119413 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91401f5a00 0x7f91400ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:36.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.755+0000 7f91457c6700 1 -- 172.21.15.43:0/98119413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91400af060 msgr2=0x7f91401f54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:36.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.755+0000 7f91457c6700 1 --2- 172.21.15.43:0/98119413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91400af060 0x7f91401f54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:36.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.755+0000 7f91457c6700 1 -- 172.21.15.43:0/98119413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c009550 con 0x7f914006ce80 2023-06-28T20:08:36.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.755+0000 7f91457c6700 1 --2- 172.21.15.43:0/98119413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f914006ce80 0x7f91401f1820 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f91300090b0 tx=0x7f9130004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:36.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.755+0000 7f9135ffb700 1 -- 172.21.15.43:0/98119413 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91300036a0 con 0x7f914006ce80 2023-06-28T20:08:36.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.755+0000 7f9135ffb700 1 -- 172.21.15.43:0/98119413 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913001f070 con 0x7f914006ce80 2023-06-28T20:08:36.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.755+0000 7f9147a2a700 1 -- 172.21.15.43:0/98119413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91300094a0 con 0x7f914006ce80 2023-06-28T20:08:36.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.755+0000 7f9135ffb700 1 -- 172.21.15.43:0/98119413 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9130003a20 con 0x7f914006ce80 2023-06-28T20:08:36.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.756+0000 7f9147a2a700 1 -- 172.21.15.43:0/98119413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f914015dbd0 con 0x7f914006ce80 2023-06-28T20:08:36.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.757+0000 7f9147a2a700 1 -- 172.21.15.43:0/98119413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f914015e500 con 0x7f914006ce80 2023-06-28T20:08:36.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.757+0000 7f9135ffb700 1 -- 172.21.15.43:0/98119413 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9130003b80 con 0x7f914006ce80 2023-06-28T20:08:36.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.757+0000 7f9135ffb700 1 --2- 172.21.15.43:0/98119413 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f912406ace0 0x7f912406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:36.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.758+0000 7f9135ffb700 1 -- 172.21.15.43:0/98119413 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9130021070 con 0x7f914006ce80 2023-06-28T20:08:36.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.758+0000 7f9144fc5700 1 --2- 172.21.15.43:0/98119413 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f912406ace0 0x7f912406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:36.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.758+0000 7f9144fc5700 1 --2- 172.21.15.43:0/98119413 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f912406ace0 0x7f912406d190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9140155f40 tx=0x7f913c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:36.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:36.762+0000 7f9135ffb700 1 -- 172.21.15.43:0/98119413 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9130055e30 con 0x7f914006ce80 2023-06-28T20:08:37.008 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.004+0000 7f9147a2a700 1 -- 172.21.15.43:0/98119413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f914006d5f0 con 0x7f914006ce80 2023-06-28T20:08:37.009 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.004+0000 7f9135ffb700 1 -- 172.21.15.43:0/98119413 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f9130059450 con 0x7f914006ce80 2023-06-28T20:08:37.010 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:37.010 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1274138939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":1274138939},{"type":"v1","addr":"172.21.15.178:6835","nonce":1274138939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3882460026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3882460026},{"type":"v1","addr":"172.21.15.178:6834","nonce":3882460026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/382468709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":382468709},{"type":"v1","addr":"172.21.15.43:6836","nonce":382468709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-28T20:08:20.073546+0000","modified":"2023-06-28T20:08:23.914623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5112},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5112":{"gid":5112,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6837/1096784002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":1096784002},{"type":"v1","addr":"172.21.15.43:6837","nonce":1096784002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-06-28T20:08:37.011 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.006+0000 7f9147a2a700 1 -- 172.21.15.43:0/98119413 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f912406ace0 msgr2=0x7f912406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:37.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.006+0000 7f9147a2a700 1 --2- 172.21.15.43:0/98119413 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f912406ace0 0x7f912406d190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9140155f40 tx=0x7f913c008040 comp rx=0 tx=0).stop 2023-06-28T20:08:37.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.006+0000 7f9147a2a700 1 -- 172.21.15.43:0/98119413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f914006ce80 msgr2=0x7f91401f1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:37.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.006+0000 7f9147a2a700 1 --2- 172.21.15.43:0/98119413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f914006ce80 0x7f91401f1820 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f91300090b0 tx=0x7f9130004750 comp rx=0 tx=0).stop 2023-06-28T20:08:37.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.006+0000 7f9147a2a700 1 -- 172.21.15.43:0/98119413 shutdown_connections 2023-06-28T20:08:37.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.006+0000 7f9147a2a700 1 --2- 172.21.15.43:0/98119413 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f912406ace0 0x7f912406d190 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.006+0000 7f9147a2a700 1 --2- 172.21.15.43:0/98119413 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f91401f5a00 0x7f91400ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.013 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.006+0000 7f9147a2a700 1 --2- 172.21.15.43:0/98119413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f914006ce80 0x7f91401f1820 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.006+0000 7f9147a2a700 1 --2- 172.21.15.43:0/98119413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91400af060 0x7f91401f54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.006+0000 7f9147a2a700 1 -- 172.21.15.43:0/98119413 >> 172.21.15.43:0/98119413 conn(0x7f91400b1720 msgr2=0x7f91400af620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:37.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.006+0000 7f9147a2a700 1 -- 172.21.15.43:0/98119413 shutdown_connections 2023-06-28T20:08:37.014 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.006+0000 7f9147a2a700 1 -- 172.21.15.43:0/98119413 wait complete. 2023-06-28T20:08:37.014 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 15 2023-06-28T20:08:37.024 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:08:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.146+0000 7f5e51e87700 1 Processor -- start 2023-06-28T20:08:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.146+0000 7f5e51e87700 1 -- start start 2023-06-28T20:08:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.147+0000 7f5e51e87700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e4c0af060 0x7f5e4c0cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.147+0000 7f5e51e87700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5e4c06ca80 0x7f5e4c069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:37.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.147+0000 7f5e51e87700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f5e4c06ce80 0x7f5e4c06a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:37.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.147+0000 7f5e51e87700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e4c071410 con 0x7f5e4c06ca80 2023-06-28T20:08:37.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.147+0000 7f5e51e87700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e4c0716b0 con 0x7f5e4c06ce80 2023-06-28T20:08:37.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.147+0000 7f5e51e87700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c071990 con 0x7f5e4c0af060 2023-06-28T20:08:37.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.147+0000 7f5e4affd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5e4c06ca80 0x7f5e4c069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:37.153 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.147+0000 7f5e4affd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5e4c06ca80 0x7f5e4c069ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55462/0 (socket says 172.21.15.43:55462) 2023-06-28T20:08:37.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.147+0000 7f5e4affd700 1 -- 172.21.15.43:0/3567662167 learned_addr learned my addr 172.21.15.43:0/3567662167 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:37.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.147+0000 7f5e4b7fe700 1 --2- 172.21.15.43:0/3567662167 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e4c0af060 0x7f5e4c0cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:37.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.147+0000 7f5e4bfff700 1 --2- 172.21.15.43:0/3567662167 >> v2:172.21.15.178:3300/0 conn(0x7f5e4c06ce80 0x7f5e4c06a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:37.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.147+0000 7f5e4affd700 1 -- 172.21.15.43:0/3567662167 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e4c0af060 msgr2=0x7f5e4c0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:37.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.147+0000 7f5e4affd700 1 --2- 172.21.15.43:0/3567662167 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e4c0af060 0x7f5e4c0cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.147+0000 7f5e4affd700 1 -- 172.21.15.43:0/3567662167 >> v2:172.21.15.178:3300/0 conn(0x7f5e4c06ce80 msgr2=0x7f5e4c06a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:37.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.147+0000 7f5e4affd700 1 --2- 172.21.15.43:0/3567662167 >> v2:172.21.15.178:3300/0 conn(0x7f5e4c06ce80 0x7f5e4c06a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.154 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.147+0000 7f5e4affd700 1 -- 172.21.15.43:0/3567662167 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c0af430 con 0x7f5e4c06ca80 2023-06-28T20:08:37.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.147+0000 7f5e4affd700 1 --2- 172.21.15.43:0/3567662167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e4c06ca80 0x7f5e4c069ef0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0ca1f0 tx=0x7f5e40009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a163ddcc416033b4 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:37.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.148+0000 7f5e4a7fc700 1 -- 172.21.15.43:0/3567662167 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40011830 con 0x7f5e4c06ca80 2023-06-28T20:08:37.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.148+0000 7f5e4a7fc700 1 -- 172.21.15.43:0/3567662167 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e40011990 con 0x7f5e4c06ca80 2023-06-28T20:08:37.155 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.148+0000 7f5e4a7fc700 1 -- 172.21.15.43:0/3567662167 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40011d20 con 0x7f5e4c06ca80 2023-06-28T20:08:37.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.148+0000 7f5e51e87700 1 -- 172.21.15.43:0/3567662167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e4c06ca80 msgr2=0x7f5e4c069ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:37.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.148+0000 7f5e51e87700 1 --2- 172.21.15.43:0/3567662167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e4c06ca80 0x7f5e4c069ef0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0ca1f0 tx=0x7f5e40009bd0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.148+0000 7f5e51e87700 1 -- 172.21.15.43:0/3567662167 shutdown_connections 2023-06-28T20:08:37.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.148+0000 7f5e51e87700 1 --2- 172.21.15.43:0/3567662167 >> v2:172.21.15.178:3300/0 conn(0x7f5e4c06ce80 0x7f5e4c06a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.148+0000 7f5e51e87700 1 --2- 172.21.15.43:0/3567662167 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e4c0af060 0x7f5e4c0cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.148+0000 7f5e51e87700 1 --2- 172.21.15.43:0/3567662167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e4c06ca80 0x7f5e4c069ef0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.148+0000 7f5e51e87700 1 -- 172.21.15.43:0/3567662167 >> 172.21.15.43:0/3567662167 conn(0x7f5e4c0b1720 msgr2=0x7f5e4c0b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:37.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.148+0000 7f5e51e87700 1 -- 172.21.15.43:0/3567662167 shutdown_connections 2023-06-28T20:08:37.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.148+0000 7f5e51e87700 1 -- 172.21.15.43:0/3567662167 wait complete. 2023-06-28T20:08:37.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.150+0000 7f5e51e87700 1 Processor -- start 2023-06-28T20:08:37.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.150+0000 7f5e51e87700 1 -- start start 2023-06-28T20:08:37.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.151+0000 7f5e51e87700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e4c06ce80 0x7f5e4c1f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:37.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.151+0000 7f5e51e87700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e4c0af060 0x7f5e4c1f5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:37.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.151+0000 7f5e51e87700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e4c1f2f20 0x7f5e4c1f5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:37.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.151+0000 7f5e51e87700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c0c9700 con 0x7f5e4c1f2f20 2023-06-28T20:08:37.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.151+0000 7f5e51e87700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c0c49c0 con 0x7f5e4c0af060 2023-06-28T20:08:37.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.151+0000 7f5e51e87700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c0c9420 con 0x7f5e4c06ce80 2023-06-28T20:08:37.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.151+0000 7f5e4bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e4c1f2f20 0x7f5e4c1f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:37.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.151+0000 7f5e4bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e4c1f2f20 0x7f5e4c1f5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55470/0 (socket says 172.21.15.43:55470) 2023-06-28T20:08:37.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.151+0000 7f5e4bfff700 1 -- 172.21.15.43:0/240092064 learned_addr learned my addr 172.21.15.43:0/240092064 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:37.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.151+0000 7f5e4bfff700 1 -- 172.21.15.43:0/240092064 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e4c06ce80 msgr2=0x7f5e4c1f4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:37.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.151+0000 7f5e4bfff700 1 --2- 172.21.15.43:0/240092064 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e4c06ce80 0x7f5e4c1f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.151+0000 7f5e4bfff700 1 -- 172.21.15.43:0/240092064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e4c0af060 msgr2=0x7f5e4c1f5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:37.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.151+0000 7f5e4affd700 1 --2- 172.21.15.43:0/240092064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e4c0af060 0x7f5e4c1f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:37.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.151+0000 7f5e4bfff700 1 --2- 172.21.15.43:0/240092064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e4c0af060 0x7f5e4c1f5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.151+0000 7f5e4bfff700 1 -- 172.21.15.43:0/240092064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e40009550 con 0x7f5e4c1f2f20 2023-06-28T20:08:37.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.152+0000 7f5e4bfff700 1 --2- 172.21.15.43:0/240092064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e4c1f2f20 0x7f5e4c1f5940 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5e4400eed0 tx=0x7f5e4400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:37.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.152+0000 7f5e33fff700 1 -- 172.21.15.43:0/240092064 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44010040 con 0x7f5e4c1f2f20 2023-06-28T20:08:37.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.152+0000 7f5e33fff700 1 -- 172.21.15.43:0/240092064 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4400bd30 con 0x7f5e4c1f2f20 2023-06-28T20:08:37.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.152+0000 7f5e51e87700 1 -- 172.21.15.43:0/240092064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4c0c4c60 con 0x7f5e4c1f2f20 2023-06-28T20:08:37.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.152+0000 7f5e33fff700 1 -- 172.21.15.43:0/240092064 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44009ea0 con 0x7f5e4c1f2f20 2023-06-28T20:08:37.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.152+0000 7f5e51e87700 1 -- 172.21.15.43:0/240092064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4c1f50d0 con 0x7f5e4c1f2f20 2023-06-28T20:08:37.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.153+0000 7f5e51e87700 1 -- 172.21.15.43:0/240092064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e4c0c5180 con 0x7f5e4c1f2f20 2023-06-28T20:08:37.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.153+0000 7f5e33fff700 1 -- 172.21.15.43:0/240092064 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5e44014640 con 0x7f5e4c1f2f20 2023-06-28T20:08:37.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.154+0000 7f5e33fff700 1 --2- 172.21.15.43:0/240092064 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5e3806ace0 0x7f5e3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:37.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.154+0000 7f5e33fff700 1 -- 172.21.15.43:0/240092064 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5e44089f50 con 0x7f5e4c1f2f20 2023-06-28T20:08:37.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.154+0000 7f5e4b7fe700 1 --2- 172.21.15.43:0/240092064 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5e3806ace0 0x7f5e3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:37.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.154+0000 7f5e4b7fe700 1 --2- 172.21.15.43:0/240092064 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5e3806ace0 0x7f5e3806d190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c006b20 tx=0x7f5e3c006ab0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:37.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.158+0000 7f5e33fff700 1 -- 172.21.15.43:0/240092064 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5e44054f70 con 0x7f5e4c1f2f20 2023-06-28T20:08:37.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.354+0000 7f5e51e87700 1 -- 172.21.15.43:0/240092064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e4c0728b0 con 0x7f5e4c1f2f20 2023-06-28T20:08:37.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.356+0000 7f5e33fff700 1 -- 172.21.15.43:0/240092064 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+18293 (secure 0 0 0) 0x7f5e44058590 con 0x7f5e4c1f2f20 2023-06-28T20:08:37.361 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:37.362 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":35,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:08:21.654904+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:08:17.811454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-06-28T20:08:19.046351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:08:37.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.358+0000 7f5e51e87700 1 -- 172.21.15.43:0/240092064 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5e3806ace0 msgr2=0x7f5e3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:37.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.358+0000 7f5e51e87700 1 --2- 172.21.15.43:0/240092064 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5e3806ace0 0x7f5e3806d190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c006b20 tx=0x7f5e3c006ab0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.358+0000 7f5e51e87700 1 -- 172.21.15.43:0/240092064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e4c1f2f20 msgr2=0x7f5e4c1f5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:37.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.358+0000 7f5e51e87700 1 --2- 172.21.15.43:0/240092064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e4c1f2f20 0x7f5e4c1f5940 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f5e4400eed0 tx=0x7f5e4400b390 comp rx=0 tx=0).stop 2023-06-28T20:08:37.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.358+0000 7f5e51e87700 1 -- 172.21.15.43:0/240092064 shutdown_connections 2023-06-28T20:08:37.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.358+0000 7f5e51e87700 1 --2- 172.21.15.43:0/240092064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e4c1f2f20 0x7f5e4c1f5940 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.358+0000 7f5e51e87700 1 --2- 172.21.15.43:0/240092064 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5e3806ace0 0x7f5e3806d190 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.358+0000 7f5e51e87700 1 --2- 172.21.15.43:0/240092064 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5e4c06ce80 0x7f5e4c1f4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.358+0000 7f5e51e87700 1 --2- 172.21.15.43:0/240092064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e4c0af060 0x7f5e4c1f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.364 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.358+0000 7f5e51e87700 1 -- 172.21.15.43:0/240092064 >> 172.21.15.43:0/240092064 conn(0x7f5e4c0b1720 msgr2=0x7f5e4c0af620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:37.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.358+0000 7f5e51e87700 1 -- 172.21.15.43:0/240092064 shutdown_connections 2023-06-28T20:08:37.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.358+0000 7f5e51e87700 1 -- 172.21.15.43:0/240092064 wait complete. 2023-06-28T20:08:37.374 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-06-28T20:08:37.375 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-28T20:08:37.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.496+0000 7f49f3e08700 1 Processor -- start 2023-06-28T20:08:37.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.497+0000 7f49f3e08700 1 -- start start 2023-06-28T20:08:37.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.497+0000 7f49f3e08700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f49ec15c550 0x7f49ec15d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:37.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.497+0000 7f49f3e08700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49ec0cded0 con 0x7f49ec06af00 2023-06-28T20:08:37.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.497+0000 7f49f3e08700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49ec161060 con 0x7f49ec0ce400 2023-06-28T20:08:37.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.497+0000 7f49f3e08700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49ec15dbe0 con 0x7f49ec15c550 2023-06-28T20:08:37.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.498+0000 7f49f1ba4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f49ec15c550 0x7f49ec15d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:37.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.498+0000 7f49f23a5700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f49ec06af00 0x7f49ec160950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44662/0 (socket says 172.21.15.43:44662) 2023-06-28T20:08:37.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.498+0000 7f49f23a5700 1 -- 172.21.15.43:0/1556247688 learned_addr learned my addr 172.21.15.43:0/1556247688 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:37.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.498+0000 7f49f0ba2700 1 -- 172.21.15.43:0/1556247688 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 745547676 0 0) 0x7f49ec0cded0 con 0x7f49ec06af00 2023-06-28T20:08:37.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.498+0000 7f49f0ba2700 1 -- 172.21.15.43:0/1556247688 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49d80035b0 con 0x7f49ec06af00 2023-06-28T20:08:37.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.498+0000 7f49f1ba4700 1 -- 172.21.15.43:0/1556247688 >> v1:172.21.15.178:6789/0 conn(0x7f49ec0ce400 legacy=0x7f49ec160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:37.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.498+0000 7f49f1ba4700 1 -- 172.21.15.43:0/1556247688 >> v1:172.21.15.43:6789/0 conn(0x7f49ec06af00 legacy=0x7f49ec160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:37.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.498+0000 7f49f1ba4700 1 -- 172.21.15.43:0/1556247688 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ec0cd2c0 con 0x7f49ec15c550 2023-06-28T20:08:37.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.498+0000 7f49f1ba4700 1 --2- 172.21.15.43:0/1556247688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec15c550 0x7f49ec15d6a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f49dc009870 tx=0x7f49dc009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975318604380d429 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:37.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.498+0000 7f49f0ba2700 1 -- 172.21.15.43:0/1556247688 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc004230 con 0x7f49ec15c550 2023-06-28T20:08:37.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.498+0000 7f49f0ba2700 1 -- 172.21.15.43:0/1556247688 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49dc004390 con 0x7f49ec15c550 2023-06-28T20:08:37.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.498+0000 7f49f0ba2700 1 -- 172.21.15.43:0/1556247688 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc0036a0 con 0x7f49ec15c550 2023-06-28T20:08:37.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.499+0000 7f49f3e08700 1 -- 172.21.15.43:0/1556247688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec15c550 msgr2=0x7f49ec15d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:37.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.499+0000 7f49f3e08700 1 --2- 172.21.15.43:0/1556247688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec15c550 0x7f49ec15d6a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f49dc009870 tx=0x7f49dc009b80 comp rx=0 tx=0).stop 2023-06-28T20:08:37.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.499+0000 7f49f3e08700 1 -- 172.21.15.43:0/1556247688 shutdown_connections 2023-06-28T20:08:37.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.499+0000 7f49f3e08700 1 --2- 172.21.15.43:0/1556247688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec15c550 0x7f49ec15d6a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.499+0000 7f49f3e08700 1 -- 172.21.15.43:0/1556247688 >> 172.21.15.43:0/1556247688 conn(0x7f49ec06c170 msgr2=0x7f49ec06c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:37.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.499+0000 7f49f3e08700 1 -- 172.21.15.43:0/1556247688 shutdown_connections 2023-06-28T20:08:37.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.499+0000 7f49f3e08700 1 -- 172.21.15.43:0/1556247688 wait complete. 2023-06-28T20:08:37.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.500+0000 7f49f3e08700 1 Processor -- start 2023-06-28T20:08:37.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.500+0000 7f49f3e08700 1 -- start start 2023-06-28T20:08:37.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.501+0000 7f49f3e08700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49ec06af00 0x7f49ec149900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:37.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.501+0000 7f49f3e08700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec0ce400 0x7f49ec0af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:37.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.501+0000 7f49f3e08700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49ec15c550 0x7f49ec0af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:37.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.501+0000 7f49f3e08700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec0c2e20 con 0x7f49ec0ce400 2023-06-28T20:08:37.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.501+0000 7f49f3e08700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec0c3100 con 0x7f49ec06af00 2023-06-28T20:08:37.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.501+0000 7f49f3e08700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec0c33e0 con 0x7f49ec15c550 2023-06-28T20:08:37.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.501+0000 7f49f13a3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec0ce400 0x7f49ec0af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:37.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.501+0000 7f49f13a3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec0ce400 0x7f49ec0af1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55480/0 (socket says 172.21.15.43:55480) 2023-06-28T20:08:37.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.501+0000 7f49f13a3700 1 -- 172.21.15.43:0/323115388 learned_addr learned my addr 172.21.15.43:0/323115388 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:37.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.501+0000 7f49f23a5700 1 --2- 172.21.15.43:0/323115388 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49ec15c550 0x7f49ec0af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:37.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.501+0000 7f49f13a3700 1 -- 172.21.15.43:0/323115388 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49ec15c550 msgr2=0x7f49ec0af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:37.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.501+0000 7f49f1ba4700 1 --2- 172.21.15.43:0/323115388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49ec06af00 0x7f49ec149900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:37.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.501+0000 7f49f13a3700 1 --2- 172.21.15.43:0/323115388 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49ec15c550 0x7f49ec0af720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.501+0000 7f49f13a3700 1 -- 172.21.15.43:0/323115388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49ec06af00 msgr2=0x7f49ec149900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:37.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.501+0000 7f49f13a3700 1 --2- 172.21.15.43:0/323115388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49ec06af00 0x7f49ec149900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.501+0000 7f49f13a3700 1 -- 172.21.15.43:0/323115388 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49dc0094a0 con 0x7f49ec0ce400 2023-06-28T20:08:37.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.501+0000 7f49f23a5700 1 --2- 172.21.15.43:0/323115388 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49ec15c550 0x7f49ec0af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:37.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.501+0000 7f49f1ba4700 1 --2- 172.21.15.43:0/323115388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49ec06af00 0x7f49ec149900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:37.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.501+0000 7f49f13a3700 1 --2- 172.21.15.43:0/323115388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec0ce400 0x7f49ec0af1e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f49e800c4d0 tx=0x7f49e800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:37.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.502+0000 7f49e27fc700 1 -- 172.21.15.43:0/323115388 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e8011790 con 0x7f49ec0ce400 2023-06-28T20:08:37.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.502+0000 7f49e27fc700 1 -- 172.21.15.43:0/323115388 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e80118f0 con 0x7f49ec0ce400 2023-06-28T20:08:37.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.502+0000 7f49f3e08700 1 -- 172.21.15.43:0/323115388 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ec0c36c0 con 0x7f49ec0ce400 2023-06-28T20:08:37.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.502+0000 7f49e27fc700 1 -- 172.21.15.43:0/323115388 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e8011ca0 con 0x7f49ec0ce400 2023-06-28T20:08:37.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.502+0000 7f49f3e08700 1 -- 172.21.15.43:0/323115388 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ec15e380 con 0x7f49ec0ce400 2023-06-28T20:08:37.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.503+0000 7f49f3e08700 1 -- 172.21.15.43:0/323115388 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49ec0cdda0 con 0x7f49ec0ce400 2023-06-28T20:08:37.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.503+0000 7f49e27fc700 1 -- 172.21.15.43:0/323115388 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f49e801c460 con 0x7f49ec0ce400 2023-06-28T20:08:37.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.503+0000 7f49e27fc700 1 --2- 172.21.15.43:0/323115388 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f49d006ace0 0x7f49d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:37.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.503+0000 7f49e27fc700 1 -- 172.21.15.43:0/323115388 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f49e8089330 con 0x7f49ec0ce400 2023-06-28T20:08:37.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.504+0000 7f49f1ba4700 1 --2- 172.21.15.43:0/323115388 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f49d006ace0 0x7f49d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:37.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.504+0000 7f49f1ba4700 1 --2- 172.21.15.43:0/323115388 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f49d006ace0 0x7f49d006d190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f49dc009bc0 tx=0x7f49dc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:37.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.508+0000 7f49e27fc700 1 -- 172.21.15.43:0/323115388 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f49e80543e0 con 0x7f49ec0ce400 2023-06-28T20:08:37.760 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.756+0000 7f49f3e08700 1 -- 172.21.15.43:0/323115388 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49ec15e5a0 con 0x7f49ec0ce400 2023-06-28T20:08:37.761 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.757+0000 7f49e27fc700 1 -- 172.21.15.43:0/323115388 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f49e8057a00 con 0x7f49ec0ce400 2023-06-28T20:08:37.761 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:37.762 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1274138939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":1274138939},{"type":"v1","addr":"172.21.15.178:6835","nonce":1274138939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3882460026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3882460026},{"type":"v1","addr":"172.21.15.178:6834","nonce":3882460026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/382468709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":382468709},{"type":"v1","addr":"172.21.15.43:6836","nonce":382468709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-28T20:08:20.073546+0000","modified":"2023-06-28T20:08:23.914623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5112},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5112":{"gid":5112,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6837/1096784002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":1096784002},{"type":"v1","addr":"172.21.15.43:6837","nonce":1096784002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-06-28T20:08:37.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.759+0000 7f49f3e08700 1 -- 172.21.15.43:0/323115388 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f49d006ace0 msgr2=0x7f49d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:37.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.759+0000 7f49f3e08700 1 --2- 172.21.15.43:0/323115388 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f49d006ace0 0x7f49d006d190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f49dc009bc0 tx=0x7f49dc008030 comp rx=0 tx=0).stop 2023-06-28T20:08:37.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.759+0000 7f49f3e08700 1 -- 172.21.15.43:0/323115388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec0ce400 msgr2=0x7f49ec0af1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:37.763 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.759+0000 7f49f3e08700 1 --2- 172.21.15.43:0/323115388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec0ce400 0x7f49ec0af1e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f49e800c4d0 tx=0x7f49e800c890 comp rx=0 tx=0).stop 2023-06-28T20:08:37.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.759+0000 7f49f3e08700 1 -- 172.21.15.43:0/323115388 shutdown_connections 2023-06-28T20:08:37.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.759+0000 7f49f3e08700 1 --2- 172.21.15.43:0/323115388 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f49d006ace0 0x7f49d006d190 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.759+0000 7f49f3e08700 1 --2- 172.21.15.43:0/323115388 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f49ec15c550 0x7f49ec0af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.759+0000 7f49f3e08700 1 --2- 172.21.15.43:0/323115388 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49ec06af00 0x7f49ec149900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.759+0000 7f49f3e08700 1 --2- 172.21.15.43:0/323115388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49ec0ce400 0x7f49ec0af1e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.759+0000 7f49f3e08700 1 -- 172.21.15.43:0/323115388 >> 172.21.15.43:0/323115388 conn(0x7f49ec06c170 msgr2=0x7f49ec15b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:37.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.759+0000 7f49f3e08700 1 -- 172.21.15.43:0/323115388 shutdown_connections 2023-06-28T20:08:37.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.759+0000 7f49f3e08700 1 -- 172.21.15.43:0/323115388 wait complete. 2023-06-28T20:08:37.765 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 15 2023-06-28T20:08:37.775 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-28T20:08:37.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.896+0000 7f232828e700 1 Processor -- start 2023-06-28T20:08:37.900 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.896+0000 7f232828e700 1 -- start start 2023-06-28T20:08:37.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.897+0000 7f232828e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23200af060 0x7f23200cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:37.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.897+0000 7f232828e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f232006a600 0x7f23200ac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:37.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.897+0000 7f232828e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2320071410 con 0x7f232006ca80 2023-06-28T20:08:37.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.897+0000 7f232828e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23200716b0 con 0x7f232006a600 2023-06-28T20:08:37.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.897+0000 7f232828e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2320071990 con 0x7f23200af060 2023-06-28T20:08:37.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.897+0000 7f2325829700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f232006ca80 0x7f2320069ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44672/0 (socket says 172.21.15.43:44672) 2023-06-28T20:08:37.907 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.897+0000 7f2325829700 1 -- 172.21.15.43:0/702765869 learned_addr learned my addr 172.21.15.43:0/702765869 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:37.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.897+0000 7f232682b700 1 --2- 172.21.15.43:0/702765869 >> v2:172.21.15.178:3300/0 conn(0x7f232006a600 0x7f23200ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:37.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.897+0000 7f232602a700 1 --2- 172.21.15.43:0/702765869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23200af060 0x7f23200cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:37.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.897+0000 7f2325028700 1 -- 172.21.15.43:0/702765869 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630981653 0 0) 0x7f2320071410 con 0x7f232006ca80 2023-06-28T20:08:37.908 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.897+0000 7f2325028700 1 -- 172.21.15.43:0/702765869 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f230c0035b0 con 0x7f232006ca80 2023-06-28T20:08:37.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.897+0000 7f232602a700 1 -- 172.21.15.43:0/702765869 >> v2:172.21.15.178:3300/0 conn(0x7f232006a600 msgr2=0x7f23200ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:37.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.897+0000 7f232602a700 1 --2- 172.21.15.43:0/702765869 >> v2:172.21.15.178:3300/0 conn(0x7f232006a600 0x7f23200ac870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.897+0000 7f232602a700 1 -- 172.21.15.43:0/702765869 >> v1:172.21.15.43:6789/0 conn(0x7f232006ca80 legacy=0x7f2320069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:37.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.897+0000 7f232602a700 1 -- 172.21.15.43:0/702765869 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23200af430 con 0x7f23200af060 2023-06-28T20:08:37.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.897+0000 7f232682b700 1 --2- 172.21.15.43:0/702765869 >> v2:172.21.15.178:3300/0 conn(0x7f232006a600 0x7f23200ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:08:37.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.897+0000 7f232602a700 1 --2- 172.21.15.43:0/702765869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23200af060 0x7f23200cf680 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f231c009840 tx=0x7f231c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8efc4dd670faf8a server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:37.909 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.898+0000 7f2325028700 1 -- 172.21.15.43:0/702765869 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c011830 con 0x7f23200af060 2023-06-28T20:08:37.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.898+0000 7f2325028700 1 -- 172.21.15.43:0/702765869 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231c011990 con 0x7f23200af060 2023-06-28T20:08:37.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.898+0000 7f2325028700 1 -- 172.21.15.43:0/702765869 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c011d00 con 0x7f23200af060 2023-06-28T20:08:37.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.898+0000 7f232828e700 1 -- 172.21.15.43:0/702765869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23200af060 msgr2=0x7f23200cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:37.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.898+0000 7f232828e700 1 --2- 172.21.15.43:0/702765869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23200af060 0x7f23200cf680 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f231c009840 tx=0x7f231c009b50 comp rx=0 tx=0).stop 2023-06-28T20:08:37.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.898+0000 7f232828e700 1 -- 172.21.15.43:0/702765869 shutdown_connections 2023-06-28T20:08:37.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.898+0000 7f232828e700 1 --2- 172.21.15.43:0/702765869 >> v2:172.21.15.178:3300/0 conn(0x7f232006a600 0x7f23200ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.898+0000 7f232828e700 1 --2- 172.21.15.43:0/702765869 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f23200af060 0x7f23200cf680 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.910 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.898+0000 7f232828e700 1 -- 172.21.15.43:0/702765869 >> 172.21.15.43:0/702765869 conn(0x7f23200b1720 msgr2=0x7f23200b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:37.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.898+0000 7f232828e700 1 -- 172.21.15.43:0/702765869 shutdown_connections 2023-06-28T20:08:37.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.898+0000 7f232828e700 1 -- 172.21.15.43:0/702765869 wait complete. 2023-06-28T20:08:37.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.900+0000 7f232828e700 1 Processor -- start 2023-06-28T20:08:37.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.900+0000 7f232828e700 1 -- start start 2023-06-28T20:08:37.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.901+0000 7f232828e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f232006a600 0x7f23200c3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:37.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.901+0000 7f232828e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23200af060 0x7f23200c43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:37.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.901+0000 7f232828e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23200c5f30 0x7f23200c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:37.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.901+0000 7f232828e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23200c4930 con 0x7f23200af060 2023-06-28T20:08:37.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.901+0000 7f232828e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23201f2320 con 0x7f23200c5f30 2023-06-28T20:08:37.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.901+0000 7f232828e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23201f2600 con 0x7f232006a600 2023-06-28T20:08:37.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.901+0000 7f2325829700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23200af060 0x7f23200c43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:37.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.901+0000 7f2325829700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23200af060 0x7f23200c43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55494/0 (socket says 172.21.15.43:55494) 2023-06-28T20:08:37.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.901+0000 7f2325829700 1 -- 172.21.15.43:0/2877167639 learned_addr learned my addr 172.21.15.43:0/2877167639 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:37.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.901+0000 7f2325829700 1 -- 172.21.15.43:0/2877167639 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f232006a600 msgr2=0x7f23200c3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:37.913 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.901+0000 7f2325829700 1 --2- 172.21.15.43:0/2877167639 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f232006a600 0x7f23200c3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.901+0000 7f2325829700 1 -- 172.21.15.43:0/2877167639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23200c5f30 msgr2=0x7f23200c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:37.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.901+0000 7f2325829700 1 --2- 172.21.15.43:0/2877167639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23200c5f30 0x7f23200c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:37.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.901+0000 7f2325829700 1 -- 172.21.15.43:0/2877167639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2318009a70 con 0x7f23200af060 2023-06-28T20:08:37.914 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.902+0000 7f2325829700 1 --2- 172.21.15.43:0/2877167639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23200af060 0x7f23200c43f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f231000c4d0 tx=0x7f231000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:37.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.902+0000 7f2316ffd700 1 -- 172.21.15.43:0/2877167639 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2310005d70 con 0x7f23200af060 2023-06-28T20:08:37.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.902+0000 7f2316ffd700 1 -- 172.21.15.43:0/2877167639 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23100070d0 con 0x7f23200af060 2023-06-28T20:08:37.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.902+0000 7f232828e700 1 -- 172.21.15.43:0/2877167639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f231c0094a0 con 0x7f23200af060 2023-06-28T20:08:37.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.902+0000 7f2316ffd700 1 -- 172.21.15.43:0/2877167639 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2310007440 con 0x7f23200af060 2023-06-28T20:08:37.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.902+0000 7f232828e700 1 -- 172.21.15.43:0/2877167639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23201f2bc0 con 0x7f23200af060 2023-06-28T20:08:37.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.903+0000 7f232828e700 1 -- 172.21.15.43:0/2877167639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23200b2a40 con 0x7f23200af060 2023-06-28T20:08:37.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.903+0000 7f2316ffd700 1 -- 172.21.15.43:0/2877167639 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f23100075a0 con 0x7f23200af060 2023-06-28T20:08:37.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.903+0000 7f2316ffd700 1 --2- 172.21.15.43:0/2877167639 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f230406ace0 0x7f230406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:37.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.904+0000 7f2316ffd700 1 -- 172.21.15.43:0/2877167639 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f23100887b0 con 0x7f23200af060 2023-06-28T20:08:37.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.904+0000 7f232602a700 1 --2- 172.21.15.43:0/2877167639 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f230406ace0 0x7f230406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:37.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.904+0000 7f232602a700 1 --2- 172.21.15.43:0/2877167639 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f230406ace0 0x7f230406d190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f231c009b90 tx=0x7f231c010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:37.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:37.908+0000 7f2316ffd700 1 -- 172.21.15.43:0/2877167639 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f23200b2a40 con 0x7f23200af060 2023-06-28T20:08:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.153+0000 7f232828e700 1 -- 172.21.15.43:0/2877167639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23200b2a40 con 0x7f23200af060 2023-06-28T20:08:38.157 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.153+0000 7f2316ffd700 1 -- 172.21.15.43:0/2877167639 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f23200b2a40 con 0x7f23200af060 2023-06-28T20:08:38.158 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:38.158 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1274138939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":1274138939},{"type":"v1","addr":"172.21.15.178:6835","nonce":1274138939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3882460026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3882460026},{"type":"v1","addr":"172.21.15.178:6834","nonce":3882460026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/382468709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":382468709},{"type":"v1","addr":"172.21.15.43:6836","nonce":382468709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-28T20:08:20.073546+0000","modified":"2023-06-28T20:08:23.914623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5112},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5112":{"gid":5112,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6837/1096784002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":1096784002},{"type":"v1","addr":"172.21.15.43:6837","nonce":1096784002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-06-28T20:08:38.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.155+0000 7f232828e700 1 -- 172.21.15.43:0/2877167639 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f230406ace0 msgr2=0x7f230406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.155+0000 7f232828e700 1 --2- 172.21.15.43:0/2877167639 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f230406ace0 0x7f230406d190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f231c009b90 tx=0x7f231c010000 comp rx=0 tx=0).stop 2023-06-28T20:08:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.155+0000 7f232828e700 1 -- 172.21.15.43:0/2877167639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23200af060 msgr2=0x7f23200c43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.155+0000 7f232828e700 1 --2- 172.21.15.43:0/2877167639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23200af060 0x7f23200c43f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f231000c4d0 tx=0x7f231000c890 comp rx=0 tx=0).stop 2023-06-28T20:08:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.155+0000 7f232828e700 1 -- 172.21.15.43:0/2877167639 shutdown_connections 2023-06-28T20:08:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.155+0000 7f232828e700 1 --2- 172.21.15.43:0/2877167639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23200c5f30 0x7f23200c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.160 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.155+0000 7f232828e700 1 --2- 172.21.15.43:0/2877167639 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f230406ace0 0x7f230406d190 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.155+0000 7f232828e700 1 --2- 172.21.15.43:0/2877167639 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f232006a600 0x7f23200c3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.155+0000 7f232828e700 1 --2- 172.21.15.43:0/2877167639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23200af060 0x7f23200c43f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.155+0000 7f232828e700 1 -- 172.21.15.43:0/2877167639 >> 172.21.15.43:0/2877167639 conn(0x7f23200b1720 msgr2=0x7f23200adf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.156+0000 7f232828e700 1 -- 172.21.15.43:0/2877167639 shutdown_connections 2023-06-28T20:08:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.156+0000 7f232828e700 1 -- 172.21.15.43:0/2877167639 wait complete. 2023-06-28T20:08:38.161 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 15 2023-06-28T20:08:38.171 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:08:38.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.292+0000 7fd8aa3ae700 1 Processor -- start 2023-06-28T20:08:38.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.292+0000 7fd8aa3ae700 1 -- start start 2023-06-28T20:08:38.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.293+0000 7fd8aa3ae700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd8a40d0630 0x7fd8a40d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:38.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.293+0000 7fd8aa3ae700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fd8a406b710 0x7fd8a40ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:38.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.293+0000 7fd8aa3ae700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a40ccf10 con 0x7fd8a40cbd40 2023-06-28T20:08:38.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.293+0000 7fd8aa3ae700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8a415cd80 con 0x7fd8a40d0630 2023-06-28T20:08:38.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.293+0000 7fd8aa3ae700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8a40d0a00 con 0x7fd8a406b710 2023-06-28T20:08:38.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.293+0000 7fd8a3fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd8a40d0630 0x7fd8a40d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:38.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.293+0000 7fd8a3fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd8a40d0630 0x7fd8a40d1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55500/0 (socket says 172.21.15.43:55500) 2023-06-28T20:08:38.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.293+0000 7fd8a3fff700 1 -- 172.21.15.43:0/661938500 learned_addr learned my addr 172.21.15.43:0/661938500 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:38.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.293+0000 7fd8a894b700 1 --2- 172.21.15.43:0/661938500 >> v2:172.21.15.178:3300/0 conn(0x7fd8a406b710 0x7fd8a40ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:38.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.293+0000 7fd8a3fff700 1 -- 172.21.15.43:0/661938500 >> v2:172.21.15.178:3300/0 conn(0x7fd8a406b710 msgr2=0x7fd8a40ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:38.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.293+0000 7fd8a3fff700 1 --2- 172.21.15.43:0/661938500 >> v2:172.21.15.178:3300/0 conn(0x7fd8a406b710 0x7fd8a40ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.293+0000 7fd8a3fff700 1 -- 172.21.15.43:0/661938500 >> v1:172.21.15.178:6789/0 conn(0x7fd8a40cbd40 legacy=0x7fd8a406b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:38.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.293+0000 7fd8a3fff700 1 -- 172.21.15.43:0/661938500 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a40cc300 con 0x7fd8a40d0630 2023-06-28T20:08:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.294+0000 7fd8a3fff700 1 --2- 172.21.15.43:0/661938500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8a40d0630 0x7fd8a40d1070 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd894006fd0 tx=0x7fd894009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727e280ea6f8bd60 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.294+0000 7fd8a2ffd700 1 -- 172.21.15.43:0/661938500 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd894005e30 con 0x7fd8a40d0630 2023-06-28T20:08:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.294+0000 7fd8a2ffd700 1 -- 172.21.15.43:0/661938500 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd894004030 con 0x7fd8a40d0630 2023-06-28T20:08:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.294+0000 7fd8a2ffd700 1 -- 172.21.15.43:0/661938500 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8940043e0 con 0x7fd8a40d0630 2023-06-28T20:08:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.294+0000 7fd8aa3ae700 1 -- 172.21.15.43:0/661938500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8a40d0630 msgr2=0x7fd8a40d1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.294+0000 7fd8aa3ae700 1 --2- 172.21.15.43:0/661938500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8a40d0630 0x7fd8a40d1070 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd894006fd0 tx=0x7fd894009bd0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.294+0000 7fd8aa3ae700 1 -- 172.21.15.43:0/661938500 shutdown_connections 2023-06-28T20:08:38.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.294+0000 7fd8aa3ae700 1 --2- 172.21.15.43:0/661938500 >> v2:172.21.15.178:3300/0 conn(0x7fd8a406b710 0x7fd8a40ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.294+0000 7fd8aa3ae700 1 --2- 172.21.15.43:0/661938500 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8a40d0630 0x7fd8a40d1070 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.294+0000 7fd8aa3ae700 1 -- 172.21.15.43:0/661938500 >> 172.21.15.43:0/661938500 conn(0x7fd8a40664b0 msgr2=0x7fd8a409a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:38.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.294+0000 7fd8aa3ae700 1 -- 172.21.15.43:0/661938500 shutdown_connections 2023-06-28T20:08:38.302 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.294+0000 7fd8aa3ae700 1 -- 172.21.15.43:0/661938500 wait complete. 2023-06-28T20:08:38.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.296+0000 7fd8aa3ae700 1 Processor -- start 2023-06-28T20:08:38.304 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.296+0000 7fd8aa3ae700 1 -- start start 2023-06-28T20:08:38.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.296+0000 7fd8aa3ae700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8a406b710 0x7fd8a40d0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:38.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.296+0000 7fd8aa3ae700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8a40cbd40 0x7fd8a41f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:38.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.296+0000 7fd8aa3ae700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8a40d0630 0x7fd8a4156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:38.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.296+0000 7fd8aa3ae700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a41e4270 con 0x7fd8a40d0630 2023-06-28T20:08:38.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.296+0000 7fd8aa3ae700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a41f6c40 con 0x7fd8a406b710 2023-06-28T20:08:38.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.296+0000 7fd8aa3ae700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a41f6680 con 0x7fd8a40cbd40 2023-06-28T20:08:38.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.297+0000 7fd8a894b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8a40d0630 0x7fd8a4156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:38.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.297+0000 7fd8a894b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8a40d0630 0x7fd8a4156f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55510/0 (socket says 172.21.15.43:55510) 2023-06-28T20:08:38.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.297+0000 7fd8a894b700 1 -- 172.21.15.43:0/1691724047 learned_addr learned my addr 172.21.15.43:0/1691724047 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:38.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.297+0000 7fd8a3fff700 1 --2- 172.21.15.43:0/1691724047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8a406b710 0x7fd8a40d0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:38.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.297+0000 7fd8a37fe700 1 --2- 172.21.15.43:0/1691724047 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8a40cbd40 0x7fd8a41f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:38.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.297+0000 7fd8a894b700 1 -- 172.21.15.43:0/1691724047 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8a40cbd40 msgr2=0x7fd8a41f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:38.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.297+0000 7fd8a894b700 1 --2- 172.21.15.43:0/1691724047 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8a40cbd40 0x7fd8a41f5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.297+0000 7fd8a894b700 1 -- 172.21.15.43:0/1691724047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8a406b710 msgr2=0x7fd8a40d0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:38.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.297+0000 7fd8a894b700 1 --2- 172.21.15.43:0/1691724047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8a406b710 0x7fd8a40d0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.297+0000 7fd8a894b700 1 -- 172.21.15.43:0/1691724047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd894009500 con 0x7fd8a40d0630 2023-06-28T20:08:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.297+0000 7fd8a894b700 1 --2- 172.21.15.43:0/1691724047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8a40d0630 0x7fd8a4156f00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00efc0 tx=0x7fd89c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.297+0000 7fd8a0ff9700 1 -- 172.21.15.43:0/1691724047 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c009b20 con 0x7fd8a40d0630 2023-06-28T20:08:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.297+0000 7fd8a0ff9700 1 -- 172.21.15.43:0/1691724047 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89c00f040 con 0x7fd8a40d0630 2023-06-28T20:08:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.297+0000 7fd8aa3ae700 1 -- 172.21.15.43:0/1691724047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a41f6960 con 0x7fd8a40d0630 2023-06-28T20:08:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.297+0000 7fd8a0ff9700 1 -- 172.21.15.43:0/1691724047 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89c00cc90 con 0x7fd8a40d0630 2023-06-28T20:08:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.297+0000 7fd8aa3ae700 1 -- 172.21.15.43:0/1691724047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a40d1420 con 0x7fd8a40d0630 2023-06-28T20:08:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.298+0000 7fd8aa3ae700 1 -- 172.21.15.43:0/1691724047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a41e4420 con 0x7fd8a40d0630 2023-06-28T20:08:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.299+0000 7fd8a0ff9700 1 -- 172.21.15.43:0/1691724047 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd89c014480 con 0x7fd8a40d0630 2023-06-28T20:08:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.299+0000 7fd8a0ff9700 1 --2- 172.21.15.43:0/1691724047 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd89006ace0 0x7fd89006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.299+0000 7fd8a0ff9700 1 -- 172.21.15.43:0/1691724047 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd89c089f40 con 0x7fd8a40d0630 2023-06-28T20:08:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.299+0000 7fd8a3fff700 1 --2- 172.21.15.43:0/1691724047 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd89006ace0 0x7fd89006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.300+0000 7fd8a3fff700 1 --2- 172.21.15.43:0/1691724047 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd89006ace0 0x7fd89006d190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd894009420 tx=0x7fd894010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.304+0000 7fd8a0ff9700 1 -- 172.21.15.43:0/1691724047 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd8a41e4420 con 0x7fd8a40d0630 2023-06-28T20:08:38.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.503+0000 7fd8aa3ae700 1 -- 172.21.15.43:0/1691724047 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8a406cee0 con 0x7fd8a40d0630 2023-06-28T20:08:38.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.504+0000 7fd8a0ff9700 1 -- 172.21.15.43:0/1691724047 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+18293 (secure 0 0 0) 0x7fd89c058a00 con 0x7fd8a40d0630 2023-06-28T20:08:38.510 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:38.510 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":35,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:08:21.654904+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-06-28T20:08:17.811454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-06-28T20:08:19.046351+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:08:38.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.507+0000 7fd8aa3ae700 1 -- 172.21.15.43:0/1691724047 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd89006ace0 msgr2=0x7fd89006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:38.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.507+0000 7fd8aa3ae700 1 --2- 172.21.15.43:0/1691724047 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd89006ace0 0x7fd89006d190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd894009420 tx=0x7fd894010000 comp rx=0 tx=0).stop 2023-06-28T20:08:38.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.507+0000 7fd8aa3ae700 1 -- 172.21.15.43:0/1691724047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8a40d0630 msgr2=0x7fd8a4156f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:38.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.507+0000 7fd8aa3ae700 1 --2- 172.21.15.43:0/1691724047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8a40d0630 0x7fd8a4156f00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd89c00efc0 tx=0x7fd89c009f60 comp rx=0 tx=0).stop 2023-06-28T20:08:38.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.507+0000 7fd8aa3ae700 1 -- 172.21.15.43:0/1691724047 shutdown_connections 2023-06-28T20:08:38.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.507+0000 7fd8aa3ae700 1 --2- 172.21.15.43:0/1691724047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8a40d0630 0x7fd8a4156f00 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.507+0000 7fd8aa3ae700 1 --2- 172.21.15.43:0/1691724047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8a406b710 0x7fd8a40d0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.507+0000 7fd8aa3ae700 1 --2- 172.21.15.43:0/1691724047 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd89006ace0 0x7fd89006d190 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.507+0000 7fd8aa3ae700 1 --2- 172.21.15.43:0/1691724047 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd8a40cbd40 0x7fd8a41f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.507+0000 7fd8aa3ae700 1 -- 172.21.15.43:0/1691724047 >> 172.21.15.43:0/1691724047 conn(0x7fd8a40664b0 msgr2=0x7fd8a406f340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:38.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.507+0000 7fd8aa3ae700 1 -- 172.21.15.43:0/1691724047 shutdown_connections 2023-06-28T20:08:38.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.507+0000 7fd8aa3ae700 1 -- 172.21.15.43:0/1691724047 wait complete. 2023-06-28T20:08:38.524 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-06-28T20:08:38.650 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.646+0000 7f6c9aa4d700 1 Processor -- start 2023-06-28T20:08:38.651 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.647+0000 7f6c9aa4d700 1 -- start start 2023-06-28T20:08:38.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.647+0000 7f6c9aa4d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6c9415cae0 0x7f6c940cf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:38.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.647+0000 7f6c9aa4d700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f6c940cfe00 0x7f6c9406cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:38.652 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.647+0000 7f6c9aa4d700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c940d0680 con 0x7f6c9406d310 2023-06-28T20:08:38.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.647+0000 7f6c9aa4d700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c9406bb20 con 0x7f6c9415cae0 2023-06-28T20:08:38.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.647+0000 7f6c9aa4d700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c9406be00 con 0x7f6c940cfe00 2023-06-28T20:08:38.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.647+0000 7f6c937fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6c9415cae0 0x7f6c940cf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:38.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.647+0000 7f6c937fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6c9415cae0 0x7f6c940cf7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55518/0 (socket says 172.21.15.43:55518) 2023-06-28T20:08:38.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.648+0000 7f6c937fe700 1 -- 172.21.15.43:0/226024680 learned_addr learned my addr 172.21.15.43:0/226024680 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:38.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.648+0000 7f6c98fea700 1 --2- 172.21.15.43:0/226024680 >> v2:172.21.15.178:3300/0 conn(0x7f6c940cfe00 0x7f6c9406cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:38.653 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.648+0000 7f6c937fe700 1 -- 172.21.15.43:0/226024680 >> v2:172.21.15.178:3300/0 conn(0x7f6c940cfe00 msgr2=0x7f6c9406cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:38.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.648+0000 7f6c937fe700 1 --2- 172.21.15.43:0/226024680 >> v2:172.21.15.178:3300/0 conn(0x7f6c940cfe00 0x7f6c9406cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.648+0000 7f6c937fe700 1 -- 172.21.15.43:0/226024680 >> v1:172.21.15.178:6789/0 conn(0x7f6c9406d310 legacy=0x7f6c9406d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:38.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.648+0000 7f6c937fe700 1 -- 172.21.15.43:0/226024680 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9406c0e0 con 0x7f6c9415cae0 2023-06-28T20:08:38.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.648+0000 7f6c98fea700 1 --2- 172.21.15.43:0/226024680 >> v2:172.21.15.178:3300/0 conn(0x7f6c940cfe00 0x7f6c9406cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:38.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.648+0000 7f6c937fe700 1 --2- 172.21.15.43:0/226024680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c9415cae0 0x7f6c940cf7e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6c940c46e0 tx=0x7f6c7c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5829c04c597a1ca5 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:38.654 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.648+0000 7f6c92ffd700 1 -- 172.21.15.43:0/226024680 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c00b870 con 0x7f6c9415cae0 2023-06-28T20:08:38.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.648+0000 7f6c92ffd700 1 -- 172.21.15.43:0/226024680 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7c00b9d0 con 0x7f6c9415cae0 2023-06-28T20:08:38.655 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.648+0000 7f6c92ffd700 1 -- 172.21.15.43:0/226024680 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c00bd60 con 0x7f6c9415cae0 2023-06-28T20:08:38.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.649+0000 7f6c9aa4d700 1 -- 172.21.15.43:0/226024680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c9415cae0 msgr2=0x7f6c940cf7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:38.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.649+0000 7f6c9aa4d700 1 --2- 172.21.15.43:0/226024680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c9415cae0 0x7f6c940cf7e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6c940c46e0 tx=0x7f6c7c009bd0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.649+0000 7f6c9aa4d700 1 -- 172.21.15.43:0/226024680 shutdown_connections 2023-06-28T20:08:38.657 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.649+0000 7f6c9aa4d700 1 --2- 172.21.15.43:0/226024680 >> v2:172.21.15.178:3300/0 conn(0x7f6c940cfe00 0x7f6c9406cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.649+0000 7f6c9aa4d700 1 --2- 172.21.15.43:0/226024680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c9415cae0 0x7f6c940cf7e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.649+0000 7f6c9aa4d700 1 -- 172.21.15.43:0/226024680 >> 172.21.15.43:0/226024680 conn(0x7f6c940664b0 msgr2=0x7f6c940d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:38.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.649+0000 7f6c9aa4d700 1 -- 172.21.15.43:0/226024680 shutdown_connections 2023-06-28T20:08:38.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.649+0000 7f6c9aa4d700 1 -- 172.21.15.43:0/226024680 wait complete. 2023-06-28T20:08:38.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.650+0000 7f6c9aa4d700 1 Processor -- start 2023-06-28T20:08:38.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.650+0000 7f6c9aa4d700 1 -- start start 2023-06-28T20:08:38.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.651+0000 7f6c9aa4d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6c9406d310 0x7f6c941f6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:38.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.651+0000 7f6c9aa4d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c940cfe00 0x7f6c9414ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:38.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.651+0000 7f6c9aa4d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c9415cae0 0x7f6c9415e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:38.660 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.651+0000 7f6c9aa4d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c940c7520 con 0x7f6c940cfe00 2023-06-28T20:08:38.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.651+0000 7f6c9aa4d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c941f3030 con 0x7f6c9415cae0 2023-06-28T20:08:38.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.651+0000 7f6c9aa4d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c941f2d50 con 0x7f6c9406d310 2023-06-28T20:08:38.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.651+0000 7f6c937fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c940cfe00 0x7f6c9414ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:38.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.651+0000 7f6c937fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c940cfe00 0x7f6c9414ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55522/0 (socket says 172.21.15.43:55522) 2023-06-28T20:08:38.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.651+0000 7f6c937fe700 1 -- 172.21.15.43:0/1287150290 learned_addr learned my addr 172.21.15.43:0/1287150290 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:38.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.651+0000 7f6c937fe700 1 -- 172.21.15.43:0/1287150290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6c9406d310 msgr2=0x7f6c941f6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:38.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.651+0000 7f6c937fe700 1 --2- 172.21.15.43:0/1287150290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6c9406d310 0x7f6c941f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.651+0000 7f6c937fe700 1 -- 172.21.15.43:0/1287150290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c9415cae0 msgr2=0x7f6c9415e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:38.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.651+0000 7f6c937fe700 1 --2- 172.21.15.43:0/1287150290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c9415cae0 0x7f6c9415e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.651+0000 7f6c937fe700 1 -- 172.21.15.43:0/1287150290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7c009550 con 0x7f6c940cfe00 2023-06-28T20:08:38.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.652+0000 7f6c937fe700 1 --2- 172.21.15.43:0/1287150290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c940cfe00 0x7f6c9414ff60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c000c00 tx=0x7f6c7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:38.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.652+0000 7f6c90ff9700 1 -- 172.21.15.43:0/1287150290 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c004520 con 0x7f6c940cfe00 2023-06-28T20:08:38.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.652+0000 7f6c90ff9700 1 -- 172.21.15.43:0/1287150290 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7c01e070 con 0x7f6c940cfe00 2023-06-28T20:08:38.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.652+0000 7f6c90ff9700 1 -- 172.21.15.43:0/1287150290 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c0038a0 con 0x7f6c940cfe00 2023-06-28T20:08:38.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.652+0000 7f6c9aa4d700 1 -- 172.21.15.43:0/1287150290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c941f5770 con 0x7f6c940cfe00 2023-06-28T20:08:38.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.652+0000 7f6c9aa4d700 1 -- 172.21.15.43:0/1287150290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c941f5d30 con 0x7f6c940cfe00 2023-06-28T20:08:38.665 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.653+0000 7f6c9aa4d700 1 -- 172.21.15.43:0/1287150290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c9406ed50 con 0x7f6c940cfe00 2023-06-28T20:08:38.666 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.654+0000 7f6c90ff9700 1 -- 172.21.15.43:0/1287150290 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6c7c003a00 con 0x7f6c940cfe00 2023-06-28T20:08:38.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.654+0000 7f6c90ff9700 1 --2- 172.21.15.43:0/1287150290 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f6c7406ace0 0x7f6c7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:38.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.654+0000 7f6c90ff9700 1 -- 172.21.15.43:0/1287150290 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6c7c020070 con 0x7f6c940cfe00 2023-06-28T20:08:38.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.654+0000 7f6c93fff700 1 --2- 172.21.15.43:0/1287150290 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f6c7406ace0 0x7f6c7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:38.667 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.655+0000 7f6c93fff700 1 --2- 172.21.15.43:0/1287150290 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f6c7406ace0 0x7f6c7406d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6c84000fb0 tx=0x7f6c84007a30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:38.668 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.658+0000 7f6c90ff9700 1 -- 172.21.15.43:0/1287150290 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6c7c056100 con 0x7f6c940cfe00 2023-06-28T20:08:38.906 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.903+0000 7f6c9aa4d700 1 -- 172.21.15.43:0/1287150290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6c940728b0 con 0x7f6c940cfe00 2023-06-28T20:08:38.907 INFO:tasks.ceph.mon.a.smithi043.stderr:2023-06-28T20:08:38.903+0000 7fdb92760700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-06-28T20:08:38.911 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.907+0000 7f6c90ff9700 1 -- 172.21.15.43:0/1287150290 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v16) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6c7c003cb0 con 0x7f6c940cfe00 2023-06-28T20:08:38.914 INFO:tasks.ceph.mds.a.smithi043.stderr: -3860> 2023-06-28T20:08:22.629+0000 7f9a71563700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-28T20:08:38.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.910+0000 7f6c9aa4d700 1 -- 172.21.15.43:0/1287150290 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f6c7406ace0 msgr2=0x7f6c7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:38.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.910+0000 7f6c9aa4d700 1 --2- 172.21.15.43:0/1287150290 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f6c7406ace0 0x7f6c7406d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6c84000fb0 tx=0x7f6c84007a30 comp rx=0 tx=0).stop 2023-06-28T20:08:38.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.910+0000 7f6c9aa4d700 1 -- 172.21.15.43:0/1287150290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c940cfe00 msgr2=0x7f6c9414ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:38.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.910+0000 7f6c9aa4d700 1 --2- 172.21.15.43:0/1287150290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c940cfe00 0x7f6c9414ff60 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c000c00 tx=0x7f6c7c004750 comp rx=0 tx=0).stop 2023-06-28T20:08:38.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.910+0000 7f6c9aa4d700 1 -- 172.21.15.43:0/1287150290 shutdown_connections 2023-06-28T20:08:38.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.910+0000 7f6c9aa4d700 1 --2- 172.21.15.43:0/1287150290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c9415cae0 0x7f6c9415e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.910+0000 7f6c9aa4d700 1 --2- 172.21.15.43:0/1287150290 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f6c7406ace0 0x7f6c7406d190 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.910+0000 7f6c9aa4d700 1 --2- 172.21.15.43:0/1287150290 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f6c9406d310 0x7f6c941f6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.910+0000 7f6c9aa4d700 1 --2- 172.21.15.43:0/1287150290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c940cfe00 0x7f6c9414ff60 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:38.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.910+0000 7f6c9aa4d700 1 -- 172.21.15.43:0/1287150290 >> 172.21.15.43:0/1287150290 conn(0x7f6c940664b0 msgr2=0x7f6c940c7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:38.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.910+0000 7f6c9aa4d700 1 -- 172.21.15.43:0/1287150290 shutdown_connections 2023-06-28T20:08:38.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:38.910+0000 7f6c9aa4d700 1 -- 172.21.15.43:0/1287150290 wait complete. 2023-06-28T20:08:38.917 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-06-28T20:08:38.927 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-06-28T20:08:38.943 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2023-06-28T20:08:39.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.053+0000 7f545663f700 1 Processor -- start 2023-06-28T20:08:39.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.053+0000 7f545663f700 1 -- start start 2023-06-28T20:08:39.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.053+0000 7f545663f700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f54501592d0 0x7f54500b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:39.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.053+0000 7f545663f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f54501596a0 0x7f54500b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:39.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.053+0000 7f545663f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5450070b40 con 0x7f5450071900 2023-06-28T20:08:39.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.053+0000 7f545663f700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5450070e20 con 0x7f54501596a0 2023-06-28T20:08:39.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.053+0000 7f545663f700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5450071100 con 0x7f54501592d0 2023-06-28T20:08:39.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.054+0000 7f544f7fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f54501596a0 0x7f54500b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:39.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.054+0000 7f5454bdc700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5450071900 0x7f545015d070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44728/0 (socket says 172.21.15.43:44728) 2023-06-28T20:08:39.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.054+0000 7f5454bdc700 1 -- 172.21.15.43:0/4213849748 learned_addr learned my addr 172.21.15.43:0/4213849748 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:39.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.054+0000 7f544ffff700 1 --2- 172.21.15.43:0/4213849748 >> v2:172.21.15.178:3300/0 conn(0x7f54501592d0 0x7f54500b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:39.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.054+0000 7f544f7fe700 1 -- 172.21.15.43:0/4213849748 >> v2:172.21.15.178:3300/0 conn(0x7f54501592d0 msgr2=0x7f54500b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:39.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.054+0000 7f544f7fe700 1 --2- 172.21.15.43:0/4213849748 >> v2:172.21.15.178:3300/0 conn(0x7f54501592d0 0x7f54500b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:39.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.054+0000 7f544f7fe700 1 -- 172.21.15.43:0/4213849748 >> v1:172.21.15.43:6789/0 conn(0x7f5450071900 legacy=0x7f545015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:39.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.054+0000 7f544f7fe700 1 -- 172.21.15.43:0/4213849748 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54500713e0 con 0x7f54501596a0 2023-06-28T20:08:39.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.054+0000 7f544effd700 1 -- 172.21.15.43:0/4213849748 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 424575511 0 0) 0x7f5450070b40 con 0x7f5450071900 2023-06-28T20:08:39.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.054+0000 7f544f7fe700 1 --2- 172.21.15.43:0/4213849748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54501596a0 0x7f54500b1f10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f54500d0bf0 tx=0x7f5444009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6c0211fc482e02 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:39.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.054+0000 7f544effd700 1 -- 172.21.15.43:0/4213849748 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5444011830 con 0x7f54501596a0 2023-06-28T20:08:39.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.054+0000 7f544effd700 1 -- 172.21.15.43:0/4213849748 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5444011990 con 0x7f54501596a0 2023-06-28T20:08:39.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.054+0000 7f544effd700 1 -- 172.21.15.43:0/4213849748 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5444011d20 con 0x7f54501596a0 2023-06-28T20:08:39.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.055+0000 7f545663f700 1 -- 172.21.15.43:0/4213849748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54501596a0 msgr2=0x7f54500b1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:39.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.055+0000 7f545663f700 1 --2- 172.21.15.43:0/4213849748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54501596a0 0x7f54500b1f10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f54500d0bf0 tx=0x7f5444009b20 comp rx=0 tx=0).stop 2023-06-28T20:08:39.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.055+0000 7f545663f700 1 -- 172.21.15.43:0/4213849748 shutdown_connections 2023-06-28T20:08:39.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.055+0000 7f545663f700 1 --2- 172.21.15.43:0/4213849748 >> v2:172.21.15.178:3300/0 conn(0x7f54501592d0 0x7f54500b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:39.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.055+0000 7f545663f700 1 --2- 172.21.15.43:0/4213849748 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54501596a0 0x7f54500b1f10 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:39.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.055+0000 7f545663f700 1 -- 172.21.15.43:0/4213849748 >> 172.21.15.43:0/4213849748 conn(0x7f545006b000 msgr2=0x7f545006b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:39.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.055+0000 7f545663f700 1 -- 172.21.15.43:0/4213849748 shutdown_connections 2023-06-28T20:08:39.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.055+0000 7f545663f700 1 -- 172.21.15.43:0/4213849748 wait complete. 2023-06-28T20:08:39.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.057+0000 7f545663f700 1 Processor -- start 2023-06-28T20:08:39.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.057+0000 7f545663f700 1 -- start start 2023-06-28T20:08:39.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.058+0000 7f545663f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5450071900 0x7f54500c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:39.068 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.058+0000 7f545663f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54501592d0 0x7f5450151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:39.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.058+0000 7f545663f700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54501596a0 0x7f545015c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:39.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.058+0000 7f545663f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54500c4c10 con 0x7f5450071900 2023-06-28T20:08:39.069 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.058+0000 7f545663f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54500c4ef0 con 0x7f54501592d0 2023-06-28T20:08:39.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.058+0000 7f545663f700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54500c51d0 con 0x7f54501596a0 2023-06-28T20:08:39.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.058+0000 7f544ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5450071900 0x7f54500c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:39.070 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.058+0000 7f544ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5450071900 0x7f54500c5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55544/0 (socket says 172.21.15.43:55544) 2023-06-28T20:08:39.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.058+0000 7f544ffff700 1 -- 172.21.15.43:0/493410826 learned_addr learned my addr 172.21.15.43:0/493410826 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:39.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.058+0000 7f5454bdc700 1 --2- 172.21.15.43:0/493410826 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54501596a0 0x7f545015c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:39.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.058+0000 7f544f7fe700 1 --2- 172.21.15.43:0/493410826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54501592d0 0x7f5450151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:39.071 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.058+0000 7f544ffff700 1 -- 172.21.15.43:0/493410826 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54501596a0 msgr2=0x7f545015c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:39.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.058+0000 7f544ffff700 1 --2- 172.21.15.43:0/493410826 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54501596a0 0x7f545015c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:39.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.058+0000 7f544ffff700 1 -- 172.21.15.43:0/493410826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54501592d0 msgr2=0x7f5450151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:39.072 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.058+0000 7f544ffff700 1 --2- 172.21.15.43:0/493410826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54501592d0 0x7f5450151810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:39.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.058+0000 7f544ffff700 1 -- 172.21.15.43:0/493410826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54440094a0 con 0x7f5450071900 2023-06-28T20:08:39.073 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.058+0000 7f544f7fe700 1 --2- 172.21.15.43:0/493410826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54501592d0 0x7f5450151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:39.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.058+0000 7f544ffff700 1 --2- 172.21.15.43:0/493410826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5450071900 0x7f54500c5cb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f544000e810 tx=0x7f544000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:39.074 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.058+0000 7f544cff9700 1 -- 172.21.15.43:0/493410826 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5440013610 con 0x7f5450071900 2023-06-28T20:08:39.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.058+0000 7f544cff9700 1 -- 172.21.15.43:0/493410826 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5440013770 con 0x7f5450071900 2023-06-28T20:08:39.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.058+0000 7f545663f700 1 -- 172.21.15.43:0/493410826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54500c54b0 con 0x7f5450071900 2023-06-28T20:08:39.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.058+0000 7f544cff9700 1 -- 172.21.15.43:0/493410826 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5440013b20 con 0x7f5450071900 2023-06-28T20:08:39.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.059+0000 7f545663f700 1 -- 172.21.15.43:0/493410826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5450151d50 con 0x7f5450071900 2023-06-28T20:08:39.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.060+0000 7f545663f700 1 -- 172.21.15.43:0/493410826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54500cc3e0 con 0x7f5450071900 2023-06-28T20:08:39.075 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.060+0000 7f544cff9700 1 -- 172.21.15.43:0/493410826 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5440013c80 con 0x7f5450071900 2023-06-28T20:08:39.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.060+0000 7f544cff9700 1 --2- 172.21.15.43:0/493410826 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f543c06ace0 0x7f543c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:39.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.061+0000 7f544f7fe700 1 --2- 172.21.15.43:0/493410826 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f543c06ace0 0x7f543c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:39.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.061+0000 7f544cff9700 1 -- 172.21.15.43:0/493410826 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f54400891b0 con 0x7f5450071900 2023-06-28T20:08:39.076 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.061+0000 7f544f7fe700 1 --2- 172.21.15.43:0/493410826 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f543c06ace0 0x7f543c06d190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f54500d0bf0 tx=0x7f5444008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:39.077 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.065+0000 7f544cff9700 1 -- 172.21.15.43:0/493410826 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5440054210 con 0x7f5450071900 2023-06-28T20:08:39.312 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.308+0000 7f545663f700 1 -- 172.21.15.43:0/493410826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5450152e50 con 0x7f5450071900 2023-06-28T20:08:39.912 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.909+0000 7f544cff9700 1 -- 172.21.15.43:0/493410826 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5440057830 con 0x7f5450071900 2023-06-28T20:08:39.915 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.912+0000 7f545663f700 1 -- 172.21.15.43:0/493410826 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f543c06ace0 msgr2=0x7f543c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:39.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.912+0000 7f545663f700 1 --2- 172.21.15.43:0/493410826 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f543c06ace0 0x7f543c06d190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f54500d0bf0 tx=0x7f5444008030 comp rx=0 tx=0).stop 2023-06-28T20:08:39.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.912+0000 7f545663f700 1 -- 172.21.15.43:0/493410826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5450071900 msgr2=0x7f54500c5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:39.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.912+0000 7f545663f700 1 --2- 172.21.15.43:0/493410826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5450071900 0x7f54500c5cb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f544000e810 tx=0x7f544000ebd0 comp rx=0 tx=0).stop 2023-06-28T20:08:39.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.912+0000 7f545663f700 1 -- 172.21.15.43:0/493410826 shutdown_connections 2023-06-28T20:08:39.916 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.912+0000 7f545663f700 1 --2- 172.21.15.43:0/493410826 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f543c06ace0 0x7f543c06d190 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:39.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.912+0000 7f545663f700 1 --2- 172.21.15.43:0/493410826 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f54501596a0 0x7f545015c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:39.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.912+0000 7f545663f700 1 --2- 172.21.15.43:0/493410826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5450071900 0x7f54500c5cb0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:39.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.912+0000 7f545663f700 1 --2- 172.21.15.43:0/493410826 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f54501592d0 0x7f5450151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:39.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.912+0000 7f545663f700 1 -- 172.21.15.43:0/493410826 >> 172.21.15.43:0/493410826 conn(0x7f545006b000 msgr2=0x7f5450152740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:39.917 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.912+0000 7f545663f700 1 -- 172.21.15.43:0/493410826 shutdown_connections 2023-06-28T20:08:39.918 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:39.912+0000 7f545663f700 1 -- 172.21.15.43:0/493410826 wait complete. 2023-06-28T20:08:39.931 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-06-28T20:08:40.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.106+0000 7fd4ea787700 1 Processor -- start 2023-06-28T20:08:40.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.106+0000 7fd4ea787700 1 -- start start 2023-06-28T20:08:40.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.106+0000 7fd4ea787700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4e415d930 0x7fd4e406a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:40.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.106+0000 7fd4ea787700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd4e406e190 0x7fd4e415cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:40.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.106+0000 7fd4ea787700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e4160330 con 0x7fd4e40aed40 2023-06-28T20:08:40.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.106+0000 7fd4ea787700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4e4160fb0 con 0x7fd4e406e190 2023-06-28T20:08:40.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.106+0000 7fd4ea787700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e40af470 con 0x7fd4e415d930 2023-06-28T20:08:40.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.106+0000 7fd4e8d24700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd4e406e190 0x7fd4e415cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:40.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.106+0000 7fd4e8d24700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd4e406e190 0x7fd4e415cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55558/0 (socket says 172.21.15.43:55558) 2023-06-28T20:08:40.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.106+0000 7fd4e8d24700 1 -- 172.21.15.43:0/2969012006 learned_addr learned my addr 172.21.15.43:0/2969012006 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:40.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.106+0000 7fd4e3fff700 1 --2- 172.21.15.43:0/2969012006 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4e415d930 0x7fd4e406a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:40.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.106+0000 7fd4e8d24700 1 -- 172.21.15.43:0/2969012006 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4e415d930 msgr2=0x7fd4e406a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:40.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.106+0000 7fd4e8d24700 1 --2- 172.21.15.43:0/2969012006 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4e415d930 0x7fd4e406a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:40.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.106+0000 7fd4e8d24700 1 -- 172.21.15.43:0/2969012006 >> v1:172.21.15.178:6789/0 conn(0x7fd4e40aed40 legacy=0x7fd4e406da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:40.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.106+0000 7fd4e8d24700 1 -- 172.21.15.43:0/2969012006 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e4160610 con 0x7fd4e406e190 2023-06-28T20:08:40.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.106+0000 7fd4e8d24700 1 --2- 172.21.15.43:0/2969012006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e406e190 0x7fd4e415cbb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009e50 tx=0x7fd4dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6020ba048b61c5 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:40.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.106+0000 7fd4e2ffd700 1 -- 172.21.15.43:0/2969012006 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc00e070 con 0x7fd4e406e190 2023-06-28T20:08:40.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.106+0000 7fd4e2ffd700 1 -- 172.21.15.43:0/2969012006 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4dc00b9c0 con 0x7fd4e406e190 2023-06-28T20:08:40.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.106+0000 7fd4e2ffd700 1 -- 172.21.15.43:0/2969012006 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc00bd70 con 0x7fd4e406e190 2023-06-28T20:08:40.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.107+0000 7fd4ea787700 1 -- 172.21.15.43:0/2969012006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e406e190 msgr2=0x7fd4e415cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:40.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.107+0000 7fd4ea787700 1 --2- 172.21.15.43:0/2969012006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e406e190 0x7fd4e415cbb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009e50 tx=0x7fd4dc00b330 comp rx=0 tx=0).stop 2023-06-28T20:08:40.113 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.107+0000 7fd4ea787700 1 -- 172.21.15.43:0/2969012006 shutdown_connections 2023-06-28T20:08:40.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.107+0000 7fd4ea787700 1 --2- 172.21.15.43:0/2969012006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e406e190 0x7fd4e415cbb0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:40.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.107+0000 7fd4ea787700 1 --2- 172.21.15.43:0/2969012006 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4e415d930 0x7fd4e406a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:40.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.107+0000 7fd4ea787700 1 -- 172.21.15.43:0/2969012006 >> 172.21.15.43:0/2969012006 conn(0x7fd4e40cd240 msgr2=0x7fd4e40cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:40.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.107+0000 7fd4ea787700 1 -- 172.21.15.43:0/2969012006 shutdown_connections 2023-06-28T20:08:40.114 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.107+0000 7fd4ea787700 1 -- 172.21.15.43:0/2969012006 wait complete. 2023-06-28T20:08:40.116 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.108+0000 7fd4ea787700 1 Processor -- start 2023-06-28T20:08:40.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.108+0000 7fd4ea787700 1 -- start start 2023-06-28T20:08:40.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.109+0000 7fd4ea787700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4e406e190 0x7fd4e4151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:40.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.109+0000 7fd4ea787700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e40aed40 0x7fd4e40affc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:40.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.109+0000 7fd4ea787700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4e415d930 0x7fd4e415edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:40.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.109+0000 7fd4ea787700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e41516b0 con 0x7fd4e40aed40 2023-06-28T20:08:40.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.109+0000 7fd4ea787700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e40b45b0 con 0x7fd4e406e190 2023-06-28T20:08:40.117 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.109+0000 7fd4ea787700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e40b4890 con 0x7fd4e415d930 2023-06-28T20:08:40.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.109+0000 7fd4e37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e40aed40 0x7fd4e40affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:40.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.109+0000 7fd4e37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e40aed40 0x7fd4e40affc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55562/0 (socket says 172.21.15.43:55562) 2023-06-28T20:08:40.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.109+0000 7fd4e37fe700 1 -- 172.21.15.43:0/750995106 learned_addr learned my addr 172.21.15.43:0/750995106 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:40.118 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.109+0000 7fd4e8d24700 1 --2- 172.21.15.43:0/750995106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4e415d930 0x7fd4e415edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:40.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.109+0000 7fd4e3fff700 1 --2- 172.21.15.43:0/750995106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4e406e190 0x7fd4e4151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:40.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.110+0000 7fd4e37fe700 1 -- 172.21.15.43:0/750995106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4e415d930 msgr2=0x7fd4e415edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:40.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.110+0000 7fd4e37fe700 1 --2- 172.21.15.43:0/750995106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4e415d930 0x7fd4e415edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:40.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.110+0000 7fd4e37fe700 1 -- 172.21.15.43:0/750995106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4e406e190 msgr2=0x7fd4e4151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:40.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.110+0000 7fd4e37fe700 1 --2- 172.21.15.43:0/750995106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4e406e190 0x7fd4e4151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:40.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.110+0000 7fd4e37fe700 1 -- 172.21.15.43:0/750995106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc009b30 con 0x7fd4e40aed40 2023-06-28T20:08:40.119 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.110+0000 7fd4e37fe700 1 --2- 172.21.15.43:0/750995106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e40aed40 0x7fd4e40affc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd4d800c4e0 tx=0x7fd4d800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:40.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.110+0000 7fd4e0ff9700 1 -- 172.21.15.43:0/750995106 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d80052d0 con 0x7fd4e40aed40 2023-06-28T20:08:40.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.110+0000 7fd4e0ff9700 1 -- 172.21.15.43:0/750995106 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d8005430 con 0x7fd4e40aed40 2023-06-28T20:08:40.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.110+0000 7fd4ea787700 1 -- 172.21.15.43:0/750995106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e40b4b70 con 0x7fd4e40aed40 2023-06-28T20:08:40.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.110+0000 7fd4e0ff9700 1 -- 172.21.15.43:0/750995106 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d80038c0 con 0x7fd4e40aed40 2023-06-28T20:08:40.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.110+0000 7fd4ea787700 1 -- 172.21.15.43:0/750995106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e40b5130 con 0x7fd4e40aed40 2023-06-28T20:08:40.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.111+0000 7fd4ea787700 1 -- 172.21.15.43:0/750995106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e40728b0 con 0x7fd4e40aed40 2023-06-28T20:08:40.120 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.111+0000 7fd4e0ff9700 1 -- 172.21.15.43:0/750995106 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd4d8003a20 con 0x7fd4e40aed40 2023-06-28T20:08:40.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.112+0000 7fd4e0ff9700 1 --2- 172.21.15.43:0/750995106 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd4d006ace0 0x7fd4d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:40.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.112+0000 7fd4e0ff9700 1 -- 172.21.15.43:0/750995106 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fd4d8088820 con 0x7fd4e40aed40 2023-06-28T20:08:40.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.112+0000 7fd4e3fff700 1 --2- 172.21.15.43:0/750995106 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd4d006ace0 0x7fd4d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:40.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.112+0000 7fd4e3fff700 1 --2- 172.21.15.43:0/750995106 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd4d006ace0 0x7fd4d006d190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4006b20 tx=0x7fd4d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:40.121 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.116+0000 7fd4e0ff9700 1 -- 172.21.15.43:0/750995106 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd4d8053900 con 0x7fd4e40aed40 2023-06-28T20:08:40.317 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:40.313+0000 7fd4ea787700 1 -- 172.21.15.43:0/750995106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd4e406bd20 con 0x7fd4e40aed40 2023-06-28T20:08:41.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.083+0000 7fd4e0ff9700 1 -- 172.21.15.43:0/750995106 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd4d8056f20 con 0x7fd4e40aed40 2023-06-28T20:08:41.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.088+0000 7fd4ea787700 1 -- 172.21.15.43:0/750995106 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd4d006ace0 msgr2=0x7fd4d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:41.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.088+0000 7fd4ea787700 1 --2- 172.21.15.43:0/750995106 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd4d006ace0 0x7fd4d006d190 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4006b20 tx=0x7fd4d4008040 comp rx=0 tx=0).stop 2023-06-28T20:08:41.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.088+0000 7fd4ea787700 1 -- 172.21.15.43:0/750995106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e40aed40 msgr2=0x7fd4e40affc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:41.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.088+0000 7fd4ea787700 1 --2- 172.21.15.43:0/750995106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e40aed40 0x7fd4e40affc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd4d800c4e0 tx=0x7fd4d800c7f0 comp rx=0 tx=0).stop 2023-06-28T20:08:41.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.089+0000 7fd4ea787700 1 -- 172.21.15.43:0/750995106 shutdown_connections 2023-06-28T20:08:41.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.089+0000 7fd4ea787700 1 --2- 172.21.15.43:0/750995106 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd4d006ace0 0x7fd4d006d190 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:41.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.089+0000 7fd4ea787700 1 --2- 172.21.15.43:0/750995106 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd4e415d930 0x7fd4e415edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:41.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.089+0000 7fd4ea787700 1 --2- 172.21.15.43:0/750995106 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4e406e190 0x7fd4e4151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:41.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.089+0000 7fd4ea787700 1 --2- 172.21.15.43:0/750995106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4e40aed40 0x7fd4e40affc0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:41.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.089+0000 7fd4ea787700 1 -- 172.21.15.43:0/750995106 >> 172.21.15.43:0/750995106 conn(0x7fd4e40cd240 msgr2=0x7fd4e406cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:41.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.089+0000 7fd4ea787700 1 -- 172.21.15.43:0/750995106 shutdown_connections 2023-06-28T20:08:41.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.090+0000 7fd4ea787700 1 -- 172.21.15.43:0/750995106 wait complete. 2023-06-28T20:08:41.097 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2023-06-28T20:08:41.108 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-06-28T20:08:41.244 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.242+0000 7fe71058d700 1 Processor -- start 2023-06-28T20:08:41.245 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.242+0000 7fe71058d700 1 -- start start 2023-06-28T20:08:41.247 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.243+0000 7fe71058d700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe7080aed40 0x7fe70806a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:41.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.243+0000 7fe71058d700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe708160330 con 0x7fe70815d930 2023-06-28T20:08:41.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.243+0000 7fe71058d700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe708160610 con 0x7fe70806e190 2023-06-28T20:08:41.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.243+0000 7fe71058d700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe708160fb0 con 0x7fe7080aed40 2023-06-28T20:08:41.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.243+0000 7fe70e329700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe70815d930 0x7fe70806da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44752/0 (socket says 172.21.15.43:44752) 2023-06-28T20:08:41.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.243+0000 7fe70e329700 1 -- 172.21.15.43:0/3178558342 learned_addr learned my addr 172.21.15.43:0/3178558342 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:41.248 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.243+0000 7fe70d327700 1 -- 172.21.15.43:0/3178558342 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 47798046 0 0) 0x7fe708160330 con 0x7fe70815d930 2023-06-28T20:08:41.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.243+0000 7fe70eb2a700 1 --2- 172.21.15.43:0/3178558342 >> v2:172.21.15.178:3300/0 conn(0x7fe7080aed40 0x7fe70806a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:41.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.243+0000 7fe70d327700 1 -- 172.21.15.43:0/3178558342 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6f40035b0 con 0x7fe70815d930 2023-06-28T20:08:41.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.243+0000 7fe70d327700 1 -- 172.21.15.43:0/3178558342 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2767866076 0 0) 0x7fe6f40035b0 con 0x7fe70815d930 2023-06-28T20:08:41.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.243+0000 7fe70d327700 1 -- 172.21.15.43:0/3178558342 >> v2:172.21.15.178:3300/0 conn(0x7fe7080aed40 msgr2=0x7fe70806a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:41.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.243+0000 7fe70d327700 1 --2- 172.21.15.43:0/3178558342 >> v2:172.21.15.178:3300/0 conn(0x7fe7080aed40 0x7fe70806a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:41.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.243+0000 7fe70d327700 1 -- 172.21.15.43:0/3178558342 >> v1:172.21.15.178:6789/0 conn(0x7fe70806e190 legacy=0x7fe70806ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:41.249 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.243+0000 7fe70d327700 1 -- 172.21.15.43:0/3178558342 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7081608f0 con 0x7fe70815d930 2023-06-28T20:08:41.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.243+0000 7fe70d327700 1 -- 172.21.15.43:0/3178558342 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7fe704001670 con 0x7fe70815d930 2023-06-28T20:08:41.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.243+0000 7fe70d327700 1 -- 172.21.15.43:0/3178558342 >> v1:172.21.15.43:6789/0 conn(0x7fe70815d930 legacy=0x7fe70806da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:41.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.243+0000 7fe70d327700 1 --2- 172.21.15.43:0/3178558342 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6f4003a00 0x7fe6f4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:41.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.243+0000 7fe70d327700 1 --2- 172.21.15.43:0/3178558342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6f4004850 0x7fe6f4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:41.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.243+0000 7fe70d327700 1 --2- 172.21.15.43:0/3178558342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6f4005320 0x7fe6f400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:41.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.243+0000 7fe70d327700 1 -- 172.21.15.43:0/3178558342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe708160fb0 con 0x7fe6f4005320 2023-06-28T20:08:41.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.243+0000 7fe70d327700 1 -- 172.21.15.43:0/3178558342 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f400c250 con 0x7fe6f4004850 2023-06-28T20:08:41.250 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.243+0000 7fe70d327700 1 -- 172.21.15.43:0/3178558342 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f400c410 con 0x7fe6f4003a00 2023-06-28T20:08:41.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.243+0000 7fe70eb2a700 1 --2- 172.21.15.43:0/3178558342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6f4005320 0x7fe6f400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:41.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.244+0000 7fe70db28700 1 --2- 172.21.15.43:0/3178558342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6f4004850 0x7fe6f4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:41.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.244+0000 7fe70e329700 1 --2- 172.21.15.43:0/3178558342 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6f4003a00 0x7fe6f4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:41.251 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.244+0000 7fe70eb2a700 1 -- 172.21.15.43:0/3178558342 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6f4003a00 msgr2=0x7fe6f4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:41.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.244+0000 7fe70eb2a700 1 --2- 172.21.15.43:0/3178558342 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6f4003a00 0x7fe6f4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:41.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.244+0000 7fe70eb2a700 1 -- 172.21.15.43:0/3178558342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6f4004850 msgr2=0x7fe6f4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:41.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.244+0000 7fe70eb2a700 1 --2- 172.21.15.43:0/3178558342 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6f4004850 0x7fe6f4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:41.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.244+0000 7fe70eb2a700 1 -- 172.21.15.43:0/3178558342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f40035b0 con 0x7fe6f4005320 2023-06-28T20:08:41.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.244+0000 7fe70eb2a700 1 -- 172.21.15.43:0/3178558342 reap_dead start 2023-06-28T20:08:41.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.244+0000 7fe70eb2a700 1 --2- 172.21.15.43:0/3178558342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6f4005320 0x7fe6f400ba10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe700009fc0 tx=0x7fe6f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45cf9c5beb0794cd server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:41.252 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.244+0000 7fe70d327700 1 -- 172.21.15.43:0/3178558342 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe700003a20 con 0x7fe6f4005320 2023-06-28T20:08:41.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.244+0000 7fe70d327700 1 -- 172.21.15.43:0/3178558342 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70000f040 con 0x7fe6f4005320 2023-06-28T20:08:41.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.244+0000 7fe70d327700 1 -- 172.21.15.43:0/3178558342 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe700003de0 con 0x7fe6f4005320 2023-06-28T20:08:41.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.245+0000 7fe71058d700 1 -- 172.21.15.43:0/3178558342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6f4005320 msgr2=0x7fe6f400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:41.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.245+0000 7fe71058d700 1 --2- 172.21.15.43:0/3178558342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6f4005320 0x7fe6f400ba10 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe700009fc0 tx=0x7fe6f4005880 comp rx=0 tx=0).stop 2023-06-28T20:08:41.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.245+0000 7fe71058d700 1 -- 172.21.15.43:0/3178558342 shutdown_connections 2023-06-28T20:08:41.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.245+0000 7fe71058d700 1 --2- 172.21.15.43:0/3178558342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6f4005320 0x7fe6f400ba10 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:41.253 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.245+0000 7fe71058d700 1 -- 172.21.15.43:0/3178558342 >> 172.21.15.43:0/3178558342 conn(0x7fe7080cd240 msgr2=0x7fe7080cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:41.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.245+0000 7fe71058d700 1 -- 172.21.15.43:0/3178558342 shutdown_connections 2023-06-28T20:08:41.254 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.245+0000 7fe71058d700 1 -- 172.21.15.43:0/3178558342 wait complete. 2023-06-28T20:08:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.246+0000 7fe71058d700 1 Processor -- start 2023-06-28T20:08:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.246+0000 7fe71058d700 1 -- start start 2023-06-28T20:08:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.247+0000 7fe71058d700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6f4005320 0x7fe70815f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.247+0000 7fe71058d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7080b3ca0 0x7fe708150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.247+0000 7fe71058d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe708154f50 0x7fe7081553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:41.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.247+0000 7fe71058d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080c4b30 con 0x7fe708154f50 2023-06-28T20:08:41.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.247+0000 7fe71058d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080c4e10 con 0x7fe7080b3ca0 2023-06-28T20:08:41.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.247+0000 7fe71058d700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080c50f0 con 0x7fe6f4005320 2023-06-28T20:08:41.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.247+0000 7fe70eb2a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe708154f50 0x7fe7081553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:41.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.247+0000 7fe70eb2a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe708154f50 0x7fe7081553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55582/0 (socket says 172.21.15.43:55582) 2023-06-28T20:08:41.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.247+0000 7fe70eb2a700 1 -- 172.21.15.43:0/3316028510 learned_addr learned my addr 172.21.15.43:0/3316028510 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:41.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.247+0000 7fe70e329700 1 --2- 172.21.15.43:0/3316028510 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6f4005320 0x7fe70815f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:41.258 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.247+0000 7fe70db28700 1 --2- 172.21.15.43:0/3316028510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7080b3ca0 0x7fe708150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:41.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.247+0000 7fe70eb2a700 1 -- 172.21.15.43:0/3316028510 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6f4005320 msgr2=0x7fe70815f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:41.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.247+0000 7fe70eb2a700 1 --2- 172.21.15.43:0/3316028510 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6f4005320 0x7fe70815f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:41.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.247+0000 7fe70eb2a700 1 -- 172.21.15.43:0/3316028510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7080b3ca0 msgr2=0x7fe708150230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:41.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.247+0000 7fe70eb2a700 1 --2- 172.21.15.43:0/3316028510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7080b3ca0 0x7fe708150230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:41.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.247+0000 7fe70eb2a700 1 -- 172.21.15.43:0/3316028510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7080c53d0 con 0x7fe708154f50 2023-06-28T20:08:41.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.247+0000 7fe70e329700 1 --2- 172.21.15.43:0/3316028510 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6f4005320 0x7fe70815f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:41.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.247+0000 7fe70eb2a700 1 --2- 172.21.15.43:0/3316028510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe708154f50 0x7fe7081553c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe70806bfe0 tx=0x7fe700009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:41.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.247+0000 7fe6faffd700 1 -- 172.21.15.43:0/3316028510 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe700003e90 con 0x7fe708154f50 2023-06-28T20:08:41.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.247+0000 7fe6faffd700 1 -- 172.21.15.43:0/3316028510 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe700004230 con 0x7fe708154f50 2023-06-28T20:08:41.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.247+0000 7fe71058d700 1 -- 172.21.15.43:0/3316028510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7080c56b0 con 0x7fe708154f50 2023-06-28T20:08:41.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.248+0000 7fe6faffd700 1 -- 172.21.15.43:0/3316028510 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7000045e0 con 0x7fe708154f50 2023-06-28T20:08:41.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.248+0000 7fe71058d700 1 -- 172.21.15.43:0/3316028510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe708150770 con 0x7fe708154f50 2023-06-28T20:08:41.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.248+0000 7fe71058d700 1 -- 172.21.15.43:0/3316028510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7080c58f0 con 0x7fe708154f50 2023-06-28T20:08:41.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.249+0000 7fe6faffd700 1 -- 172.21.15.43:0/3316028510 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe700004740 con 0x7fe708154f50 2023-06-28T20:08:41.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.249+0000 7fe6faffd700 1 --2- 172.21.15.43:0/3316028510 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe6ec06ace0 0x7fe6ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:41.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.249+0000 7fe6faffd700 1 -- 172.21.15.43:0/3316028510 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5710+0+0 (secure 0 0 0) 0x7fe70008a520 con 0x7fe708154f50 2023-06-28T20:08:41.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.249+0000 7fe70e329700 1 --2- 172.21.15.43:0/3316028510 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe6ec06ace0 0x7fe6ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:41.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.250+0000 7fe70e329700 1 --2- 172.21.15.43:0/3316028510 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe6ec06ace0 0x7fe6ec06d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe704007c30 tx=0x7fe704009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:41.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.254+0000 7fe6faffd700 1 -- 172.21.15.43:0/3316028510 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe7080c58f0 con 0x7fe708154f50 2023-06-28T20:08:41.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:41.453+0000 7fe71058d700 1 -- 172.21.15.43:0/3316028510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe7080728b0 con 0x7fe708154f50 2023-06-28T20:08:42.094 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.091+0000 7fe6faffd700 1 -- 172.21.15.43:0/3316028510 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe700059250 con 0x7fe708154f50 2023-06-28T20:08:42.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.095+0000 7fe71058d700 1 -- 172.21.15.43:0/3316028510 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe6ec06ace0 msgr2=0x7fe6ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:42.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.095+0000 7fe71058d700 1 --2- 172.21.15.43:0/3316028510 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe6ec06ace0 0x7fe6ec06d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe704007c30 tx=0x7fe704009040 comp rx=0 tx=0).stop 2023-06-28T20:08:42.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.095+0000 7fe71058d700 1 -- 172.21.15.43:0/3316028510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe708154f50 msgr2=0x7fe7081553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:42.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.095+0000 7fe71058d700 1 --2- 172.21.15.43:0/3316028510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe708154f50 0x7fe7081553c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe70806bfe0 tx=0x7fe700009750 comp rx=0 tx=0).stop 2023-06-28T20:08:42.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.095+0000 7fe71058d700 1 -- 172.21.15.43:0/3316028510 shutdown_connections 2023-06-28T20:08:42.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.095+0000 7fe71058d700 1 --2- 172.21.15.43:0/3316028510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe708154f50 0x7fe7081553c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:42.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.095+0000 7fe71058d700 1 --2- 172.21.15.43:0/3316028510 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe6ec06ace0 0x7fe6ec06d190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:42.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.095+0000 7fe71058d700 1 --2- 172.21.15.43:0/3316028510 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe6f4005320 0x7fe70815f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:42.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.095+0000 7fe71058d700 1 --2- 172.21.15.43:0/3316028510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7080b3ca0 0x7fe708150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:42.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.095+0000 7fe71058d700 1 -- 172.21.15.43:0/3316028510 >> 172.21.15.43:0/3316028510 conn(0x7fe7080cd240 msgr2=0x7fe7080ad5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:42.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.095+0000 7fe71058d700 1 -- 172.21.15.43:0/3316028510 shutdown_connections 2023-06-28T20:08:42.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.095+0000 7fe71058d700 1 -- 172.21.15.43:0/3316028510 wait complete. 2023-06-28T20:08:42.105 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2023-06-28T20:08:42.115 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-06-28T20:08:42.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.253+0000 7f65732e8700 1 Processor -- start 2023-06-28T20:08:42.256 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.253+0000 7f65732e8700 1 -- start start 2023-06-28T20:08:42.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.254+0000 7f65732e8700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f656c0af060 0x7f656c160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:42.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.254+0000 7f65732e8700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f656c06ccc0 0x7f656c0b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:42.259 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.254+0000 7f65732e8700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656c160250 con 0x7f656c0aec90 2023-06-28T20:08:42.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.254+0000 7f65732e8700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f656c15cb30 con 0x7f656c0af060 2023-06-28T20:08:42.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.254+0000 7f65732e8700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f656c071410 con 0x7f656c06ccc0 2023-06-28T20:08:42.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.254+0000 7f6570883700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f656c0af060 0x7f656c160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:42.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.254+0000 7f6571084700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f656c0aec90 0x7f656c070bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44772/0 (socket says 172.21.15.43:44772) 2023-06-28T20:08:42.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.254+0000 7f6571084700 1 -- 172.21.15.43:0/1726346204 learned_addr learned my addr 172.21.15.43:0/1726346204 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.255+0000 7f6571885700 1 --2- 172.21.15.43:0/1726346204 >> v2:172.21.15.178:3300/0 conn(0x7f656c06ccc0 0x7f656c0b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.255+0000 7f6570883700 1 -- 172.21.15.43:0/1726346204 >> v2:172.21.15.178:3300/0 conn(0x7f656c06ccc0 msgr2=0x7f656c0b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:42.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.255+0000 7f6570883700 1 --2- 172.21.15.43:0/1726346204 >> v2:172.21.15.178:3300/0 conn(0x7f656c06ccc0 0x7f656c0b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:42.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.255+0000 7f6563fff700 1 -- 172.21.15.43:0/1726346204 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3438637972 0 0) 0x7f656c160250 con 0x7f656c0aec90 2023-06-28T20:08:42.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.255+0000 7f6570883700 1 -- 172.21.15.43:0/1726346204 >> v1:172.21.15.43:6789/0 conn(0x7f656c0aec90 legacy=0x7f656c070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:42.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.255+0000 7f6570883700 1 -- 172.21.15.43:0/1726346204 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c1604f0 con 0x7f656c0af060 2023-06-28T20:08:42.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.255+0000 7f6570883700 1 --2- 172.21.15.43:0/1726346204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f656c0af060 0x7f656c160cf0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f656c0cd8a0 tx=0x7f6568009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc249003984ce49 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.255+0000 7f6563fff700 1 -- 172.21.15.43:0/1726346204 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568011830 con 0x7f656c0af060 2023-06-28T20:08:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.255+0000 7f6563fff700 1 -- 172.21.15.43:0/1726346204 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6568011990 con 0x7f656c0af060 2023-06-28T20:08:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.255+0000 7f6563fff700 1 -- 172.21.15.43:0/1726346204 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568011d20 con 0x7f656c0af060 2023-06-28T20:08:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.255+0000 7f65732e8700 1 -- 172.21.15.43:0/1726346204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f656c0af060 msgr2=0x7f656c160cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.255+0000 7f65732e8700 1 --2- 172.21.15.43:0/1726346204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f656c0af060 0x7f656c160cf0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f656c0cd8a0 tx=0x7f6568009b20 comp rx=0 tx=0).stop 2023-06-28T20:08:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.255+0000 7f65732e8700 1 -- 172.21.15.43:0/1726346204 shutdown_connections 2023-06-28T20:08:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.256+0000 7f65732e8700 1 --2- 172.21.15.43:0/1726346204 >> v2:172.21.15.178:3300/0 conn(0x7f656c06ccc0 0x7f656c0b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.256+0000 7f65732e8700 1 --2- 172.21.15.43:0/1726346204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f656c0af060 0x7f656c160cf0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.256+0000 7f65732e8700 1 -- 172.21.15.43:0/1726346204 >> 172.21.15.43:0/1726346204 conn(0x7f656c06b000 msgr2=0x7f656c06b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.256+0000 7f65732e8700 1 -- 172.21.15.43:0/1726346204 shutdown_connections 2023-06-28T20:08:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.256+0000 7f65732e8700 1 -- 172.21.15.43:0/1726346204 wait complete. 2023-06-28T20:08:42.267 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.257+0000 7f65732e8700 1 Processor -- start 2023-06-28T20:08:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.257+0000 7f65732e8700 1 -- start start 2023-06-28T20:08:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.258+0000 7f65732e8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f656c06ccc0 0x7f656c0c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.258+0000 7f65732e8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f656c0aec90 0x7f656c0b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:42.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.258+0000 7f65732e8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f656c0af060 0x7f656c0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:42.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.258+0000 7f65732e8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c0ca720 con 0x7f656c06ccc0 2023-06-28T20:08:42.269 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.258+0000 7f65732e8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c0caa00 con 0x7f656c0aec90 2023-06-28T20:08:42.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.258+0000 7f65732e8700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656c0cace0 con 0x7f656c0af060 2023-06-28T20:08:42.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.258+0000 7f6571084700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f656c06ccc0 0x7f656c0c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:42.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.258+0000 7f6571084700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f656c06ccc0 0x7f656c0c79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55592/0 (socket says 172.21.15.43:55592) 2023-06-28T20:08:42.270 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.258+0000 7f6571084700 1 -- 172.21.15.43:0/1557917132 learned_addr learned my addr 172.21.15.43:0/1557917132 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:42.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.259+0000 7f6570883700 1 --2- 172.21.15.43:0/1557917132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f656c0aec90 0x7f656c0b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:42.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.259+0000 7f6571885700 1 --2- 172.21.15.43:0/1557917132 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f656c0af060 0x7f656c0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:42.271 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.259+0000 7f6571084700 1 -- 172.21.15.43:0/1557917132 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f656c0af060 msgr2=0x7f656c0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.259+0000 7f6571084700 1 --2- 172.21.15.43:0/1557917132 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f656c0af060 0x7f656c0cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.259+0000 7f6571084700 1 -- 172.21.15.43:0/1557917132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f656c0aec90 msgr2=0x7f656c0b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:42.272 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.259+0000 7f6571084700 1 --2- 172.21.15.43:0/1557917132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f656c0aec90 0x7f656c0b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:42.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.259+0000 7f6571084700 1 -- 172.21.15.43:0/1557917132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65680094a0 con 0x7f656c06ccc0 2023-06-28T20:08:42.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.259+0000 7f6571885700 1 --2- 172.21.15.43:0/1557917132 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f656c0af060 0x7f656c0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:42.273 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.259+0000 7f6571084700 1 --2- 172.21.15.43:0/1557917132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f656c06ccc0 0x7f656c0c79d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f655c0011e0 tx=0x7f655c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:42.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.259+0000 7f6561ffb700 1 -- 172.21.15.43:0/1557917132 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c004c80 con 0x7f656c06ccc0 2023-06-28T20:08:42.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.259+0000 7f6561ffb700 1 -- 172.21.15.43:0/1557917132 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655c004e00 con 0x7f656c06ccc0 2023-06-28T20:08:42.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.259+0000 7f65732e8700 1 -- 172.21.15.43:0/1557917132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f656c0cafc0 con 0x7f656c06ccc0 2023-06-28T20:08:42.274 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.259+0000 7f6561ffb700 1 -- 172.21.15.43:0/1557917132 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c007310 con 0x7f656c06ccc0 2023-06-28T20:08:42.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.259+0000 7f65732e8700 1 -- 172.21.15.43:0/1557917132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f656c0c7f10 con 0x7f656c06ccc0 2023-06-28T20:08:42.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.260+0000 7f65732e8700 1 -- 172.21.15.43:0/1557917132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f656c0c8100 con 0x7f656c06ccc0 2023-06-28T20:08:42.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.260+0000 7f6561ffb700 1 -- 172.21.15.43:0/1557917132 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f655c007470 con 0x7f656c06ccc0 2023-06-28T20:08:42.275 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.261+0000 7f6561ffb700 1 --2- 172.21.15.43:0/1557917132 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f655006ace0 0x7f655006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.261+0000 7f6561ffb700 1 -- 172.21.15.43:0/1557917132 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f655c088f20 con 0x7f656c06ccc0 2023-06-28T20:08:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.261+0000 7f6570883700 1 --2- 172.21.15.43:0/1557917132 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f655006ace0 0x7f655006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:42.276 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.261+0000 7f6570883700 1 --2- 172.21.15.43:0/1557917132 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f655006ace0 0x7f655006d190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f65680090b0 tx=0x7f656801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:42.277 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.265+0000 7f6561ffb700 1 -- 172.21.15.43:0/1557917132 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f655c0542d0 con 0x7f656c06ccc0 2023-06-28T20:08:42.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.463+0000 7f65732e8700 1 -- 172.21.15.43:0/1557917132 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f656c0728b0 con 0x7f656c06ccc0 2023-06-28T20:08:42.469 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.467+0000 7f6561ffb700 1 -- 172.21.15.43:0/1557917132 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f655c0578f0 con 0x7f656c06ccc0 2023-06-28T20:08:42.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.469+0000 7f65732e8700 1 -- 172.21.15.43:0/1557917132 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f655006ace0 msgr2=0x7f655006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:42.472 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.469+0000 7f65732e8700 1 --2- 172.21.15.43:0/1557917132 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f655006ace0 0x7f655006d190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f65680090b0 tx=0x7f656801c040 comp rx=0 tx=0).stop 2023-06-28T20:08:42.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.469+0000 7f65732e8700 1 -- 172.21.15.43:0/1557917132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f656c06ccc0 msgr2=0x7f656c0c79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:42.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.469+0000 7f65732e8700 1 --2- 172.21.15.43:0/1557917132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f656c06ccc0 0x7f656c0c79d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f655c0011e0 tx=0x7f655c00cae0 comp rx=0 tx=0).stop 2023-06-28T20:08:42.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.469+0000 7f65732e8700 1 -- 172.21.15.43:0/1557917132 shutdown_connections 2023-06-28T20:08:42.473 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.469+0000 7f65732e8700 1 --2- 172.21.15.43:0/1557917132 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f655006ace0 0x7f655006d190 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:42.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.469+0000 7f65732e8700 1 --2- 172.21.15.43:0/1557917132 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f656c0af060 0x7f656c0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:42.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.469+0000 7f65732e8700 1 --2- 172.21.15.43:0/1557917132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f656c06ccc0 0x7f656c0c79d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:42.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.469+0000 7f65732e8700 1 --2- 172.21.15.43:0/1557917132 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f656c0aec90 0x7f656c0b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:42.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.469+0000 7f65732e8700 1 -- 172.21.15.43:0/1557917132 >> 172.21.15.43:0/1557917132 conn(0x7f656c06b000 msgr2=0x7f656c15ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:42.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.469+0000 7f65732e8700 1 -- 172.21.15.43:0/1557917132 shutdown_connections 2023-06-28T20:08:42.474 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.469+0000 7f65732e8700 1 -- 172.21.15.43:0/1557917132 wait complete. 2023-06-28T20:08:42.484 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values' 2023-06-28T20:08:42.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.607+0000 7f0ca5160700 1 Processor -- start 2023-06-28T20:08:42.611 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.607+0000 7f0ca5160700 1 -- start start 2023-06-28T20:08:42.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.608+0000 7f0ca5160700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ca00cb5c0 0x7f0ca015b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:42.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.608+0000 7f0ca5160700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca015e250 con 0x7f0ca00cb1f0 2023-06-28T20:08:42.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.608+0000 7f0ca5160700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca015e530 con 0x7f0ca00cc3d0 2023-06-28T20:08:42.614 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.608+0000 7f0ca5160700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca015e040 con 0x7f0ca00cb5c0 2023-06-28T20:08:42.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.608+0000 7f0c9ed9d700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0ca00cb1f0 0x7f0ca00cbcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44786/0 (socket says 172.21.15.43:44786) 2023-06-28T20:08:42.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.608+0000 7f0c9ed9d700 1 -- 172.21.15.43:0/2400601451 learned_addr learned my addr 172.21.15.43:0/2400601451 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:42.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.609+0000 7f0c9f59e700 1 --2- 172.21.15.43:0/2400601451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ca00cb5c0 0x7f0ca015b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:42.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.609+0000 7f0c9dd9b700 1 -- 172.21.15.43:0/2400601451 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3705753698 0 0) 0x7f0ca015e250 con 0x7f0ca00cb1f0 2023-06-28T20:08:42.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.609+0000 7f0c9dd9b700 1 -- 172.21.15.43:0/2400601451 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c8c0035b0 con 0x7f0ca00cb1f0 2023-06-28T20:08:42.615 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.609+0000 7f0c9dd9b700 1 -- 172.21.15.43:0/2400601451 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1039464877 0 0) 0x7f0ca015e530 con 0x7f0ca00cc3d0 2023-06-28T20:08:42.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.609+0000 7f0c9dd9b700 1 -- 172.21.15.43:0/2400601451 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ca015e250 con 0x7f0ca00cc3d0 2023-06-28T20:08:42.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.609+0000 7f0c9dd9b700 1 -- 172.21.15.43:0/2400601451 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2141325457 0 0) 0x7f0c8c0035b0 con 0x7f0ca00cb1f0 2023-06-28T20:08:42.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.609+0000 7f0c9dd9b700 1 -- 172.21.15.43:0/2400601451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ca00cb5c0 msgr2=0x7f0ca015b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:42.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.609+0000 7f0c9dd9b700 1 --2- 172.21.15.43:0/2400601451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ca00cb5c0 0x7f0ca015b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:42.616 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.609+0000 7f0c9dd9b700 1 -- 172.21.15.43:0/2400601451 >> v1:172.21.15.178:6789/0 conn(0x7f0ca00cc3d0 legacy=0x7f0ca0071070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:42.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.609+0000 7f0c9dd9b700 1 -- 172.21.15.43:0/2400601451 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca015e810 con 0x7f0ca00cb1f0 2023-06-28T20:08:42.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.609+0000 7f0c9dd9b700 1 -- 172.21.15.43:0/2400601451 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f0c90001670 con 0x7f0ca00cb1f0 2023-06-28T20:08:42.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.609+0000 7f0c9dd9b700 1 -- 172.21.15.43:0/2400601451 >> v1:172.21.15.43:6789/0 conn(0x7f0ca00cb1f0 legacy=0x7f0ca00cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:42.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.609+0000 7f0c9dd9b700 1 --2- 172.21.15.43:0/2400601451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ca00cb5c0 0x7f0c8c003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:42.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.609+0000 7f0c9dd9b700 1 --2- 172.21.15.43:0/2400601451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c8c004120 0x7f0c8c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:42.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.609+0000 7f0c9dd9b700 1 --2- 172.21.15.43:0/2400601451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c8c004d90 0x7f0c8c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:42.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.609+0000 7f0c9dd9b700 1 -- 172.21.15.43:0/2400601451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca015e040 con 0x7f0c8c004120 2023-06-28T20:08:42.617 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.609+0000 7f0c9dd9b700 1 -- 172.21.15.43:0/2400601451 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c00bba0 con 0x7f0c8c004d90 2023-06-28T20:08:42.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.609+0000 7f0c9dd9b700 1 -- 172.21.15.43:0/2400601451 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c00bd30 con 0x7f0ca00cb5c0 2023-06-28T20:08:42.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.609+0000 7f0c9e59c700 1 --2- 172.21.15.43:0/2400601451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c8c004120 0x7f0c8c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:42.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.609+0000 7f0c9ed9d700 1 --2- 172.21.15.43:0/2400601451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ca00cb5c0 0x7f0c8c003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:42.618 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.609+0000 7f0c9f59e700 1 --2- 172.21.15.43:0/2400601451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c8c004d90 0x7f0c8c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:42.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.609+0000 7f0c9e59c700 1 -- 172.21.15.43:0/2400601451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ca00cb5c0 msgr2=0x7f0c8c003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:42.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.609+0000 7f0c9e59c700 1 --2- 172.21.15.43:0/2400601451 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0ca00cb5c0 0x7f0c8c003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:42.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.610+0000 7f0c9e59c700 1 -- 172.21.15.43:0/2400601451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c8c004d90 msgr2=0x7f0c8c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:42.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.610+0000 7f0c9e59c700 1 --2- 172.21.15.43:0/2400601451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0c8c004d90 0x7f0c8c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:42.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.610+0000 7f0c9e59c700 1 -- 172.21.15.43:0/2400601451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8c0035b0 con 0x7f0c8c004120 2023-06-28T20:08:42.619 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.610+0000 7f0c9e59c700 1 --2- 172.21.15.43:0/2400601451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c8c004120 0x7f0c8c004850 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0c88001980 tx=0x7f0c8c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b5d95cc2ce8066 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:42.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.610+0000 7f0c9dd9b700 1 -- 172.21.15.43:0/2400601451 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88001470 con 0x7f0c8c004120 2023-06-28T20:08:42.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.610+0000 7f0c9dd9b700 1 -- 172.21.15.43:0/2400601451 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c880052d0 con 0x7f0c8c004120 2023-06-28T20:08:42.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.610+0000 7f0c9dd9b700 1 -- 172.21.15.43:0/2400601451 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c88003680 con 0x7f0c8c004120 2023-06-28T20:08:42.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.610+0000 7f0ca5160700 1 -- 172.21.15.43:0/2400601451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c8c004120 msgr2=0x7f0c8c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:42.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.610+0000 7f0ca5160700 1 --2- 172.21.15.43:0/2400601451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c8c004120 0x7f0c8c004850 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0c88001980 tx=0x7f0c8c005880 comp rx=0 tx=0).stop 2023-06-28T20:08:42.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.610+0000 7f0c9f59e700 1 -- 172.21.15.43:0/2400601451 reap_dead start 2023-06-28T20:08:42.620 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.610+0000 7f0ca5160700 1 -- 172.21.15.43:0/2400601451 shutdown_connections 2023-06-28T20:08:42.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.610+0000 7f0ca5160700 1 -- 172.21.15.43:0/2400601451 >> 172.21.15.43:0/2400601451 conn(0x7f0ca006c170 msgr2=0x7f0ca006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:42.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.610+0000 7f0ca5160700 1 -- 172.21.15.43:0/2400601451 shutdown_connections 2023-06-28T20:08:42.621 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.610+0000 7f0ca5160700 1 -- 172.21.15.43:0/2400601451 wait complete. 2023-06-28T20:08:42.623 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.612+0000 7f0ca5160700 1 Processor -- start 2023-06-28T20:08:42.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.612+0000 7f0ca5160700 1 -- start start 2023-06-28T20:08:42.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.613+0000 7f0ca5160700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c8c004120 0x7f0ca01efff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:42.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.613+0000 7f0ca5160700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ca00b4da0 0x7f0ca01f2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:42.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.613+0000 7f0ca5160700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ca00c6e70 0x7f0ca00c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:42.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.613+0000 7f0ca5160700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca00c76d0 con 0x7f0ca00c6e70 2023-06-28T20:08:42.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.613+0000 7f0ca5160700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca00b21f0 con 0x7f0ca00b4da0 2023-06-28T20:08:42.624 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.613+0000 7f0ca5160700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca00b24d0 con 0x7f0c8c004120 2023-06-28T20:08:42.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.613+0000 7f0c9f59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ca00c6e70 0x7f0ca00c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:42.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.613+0000 7f0c9f59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ca00c6e70 0x7f0ca00c8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55616/0 (socket says 172.21.15.43:55616) 2023-06-28T20:08:42.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.613+0000 7f0c9f59e700 1 -- 172.21.15.43:0/3388521552 learned_addr learned my addr 172.21.15.43:0/3388521552 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:42.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.613+0000 7f0c9e59c700 1 --2- 172.21.15.43:0/3388521552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ca00b4da0 0x7f0ca01f2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:42.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.613+0000 7f0c9ed9d700 1 --2- 172.21.15.43:0/3388521552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c8c004120 0x7f0ca01efff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:42.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.613+0000 7f0c9f59e700 1 -- 172.21.15.43:0/3388521552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c8c004120 msgr2=0x7f0ca01efff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:42.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.613+0000 7f0c9f59e700 1 --2- 172.21.15.43:0/3388521552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c8c004120 0x7f0ca01efff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:42.625 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.613+0000 7f0c9f59e700 1 -- 172.21.15.43:0/3388521552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ca00b4da0 msgr2=0x7f0ca01f2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:42.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.613+0000 7f0c9f59e700 1 --2- 172.21.15.43:0/3388521552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ca00b4da0 0x7f0ca01f2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:42.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.613+0000 7f0c9f59e700 1 -- 172.21.15.43:0/3388521552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca00b27b0 con 0x7f0ca00c6e70 2023-06-28T20:08:42.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.613+0000 7f0c9ed9d700 1 --2- 172.21.15.43:0/3388521552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c8c004120 0x7f0ca01efff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:42.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.613+0000 7f0c9f59e700 1 --2- 172.21.15.43:0/3388521552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ca00c6e70 0x7f0ca00c8400 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00cf1c0 tx=0x7f0c98011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:42.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.614+0000 7f0c977fe700 1 -- 172.21.15.43:0/3388521552 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9800e040 con 0x7f0ca00c6e70 2023-06-28T20:08:42.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.614+0000 7f0c977fe700 1 -- 172.21.15.43:0/3388521552 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9800f040 con 0x7f0ca00c6e70 2023-06-28T20:08:42.626 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.614+0000 7f0ca5160700 1 -- 172.21.15.43:0/3388521552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca00b2a90 con 0x7f0ca00c6e70 2023-06-28T20:08:42.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.614+0000 7f0c977fe700 1 -- 172.21.15.43:0/3388521552 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c9800fd00 con 0x7f0ca00c6e70 2023-06-28T20:08:42.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.614+0000 7f0ca5160700 1 -- 172.21.15.43:0/3388521552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca00c8940 con 0x7f0ca00c6e70 2023-06-28T20:08:42.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.615+0000 7f0ca5160700 1 -- 172.21.15.43:0/3388521552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca00b5170 con 0x7f0ca00c6e70 2023-06-28T20:08:42.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.615+0000 7f0c977fe700 1 -- 172.21.15.43:0/3388521552 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0c98015430 con 0x7f0ca00c6e70 2023-06-28T20:08:42.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.616+0000 7f0c977fe700 1 --2- 172.21.15.43:0/3388521552 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0c8006ace0 0x7f0c8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:42.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.616+0000 7f0c977fe700 1 -- 172.21.15.43:0/3388521552 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0c980898a0 con 0x7f0ca00c6e70 2023-06-28T20:08:42.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.616+0000 7f0c9ed9d700 1 --2- 172.21.15.43:0/3388521552 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0c8006ace0 0x7f0c8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:42.627 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.616+0000 7f0c9ed9d700 1 --2- 172.21.15.43:0/3388521552 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0c8006ace0 0x7f0c8006d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0c90001200 tx=0x7f0c90009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:42.628 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.620+0000 7f0c977fe700 1 -- 172.21.15.43:0/3388521552 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0c98058820 con 0x7f0ca00c6e70 2023-06-28T20:08:42.903 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:42.900+0000 7f0ca5160700 1 -- 172.21.15.43:0/3388521552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]} v 0) v1 -- 0x7f0ca00728b0 con 0x7f0ca00c6e70 2023-06-28T20:08:43.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:43.093+0000 7f0c977fe700 1 -- 172.21.15.43:0/3388521552 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]}]=0 v95) v1 ==== 160+0+0 (secure 0 0 0) 0x7f0c980583b0 con 0x7f0ca00c6e70 2023-06-28T20:08:43.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:43.095+0000 7f0ca5160700 1 -- 172.21.15.43:0/3388521552 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0c8006ace0 msgr2=0x7f0c8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:43.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:43.095+0000 7f0ca5160700 1 --2- 172.21.15.43:0/3388521552 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0c8006ace0 0x7f0c8006d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0c90001200 tx=0x7f0c90009040 comp rx=0 tx=0).stop 2023-06-28T20:08:43.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:43.095+0000 7f0ca5160700 1 -- 172.21.15.43:0/3388521552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ca00c6e70 msgr2=0x7f0ca00c8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:43.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:43.095+0000 7f0ca5160700 1 --2- 172.21.15.43:0/3388521552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ca00c6e70 0x7f0ca00c8400 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00cf1c0 tx=0x7f0c98011ed0 comp rx=0 tx=0).stop 2023-06-28T20:08:43.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:43.095+0000 7f0ca5160700 1 -- 172.21.15.43:0/3388521552 shutdown_connections 2023-06-28T20:08:43.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:43.095+0000 7f0ca5160700 1 --2- 172.21.15.43:0/3388521552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ca00c6e70 0x7f0ca00c8400 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:43.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:43.095+0000 7f0ca5160700 1 --2- 172.21.15.43:0/3388521552 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0c8006ace0 0x7f0c8006d190 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:43.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:43.095+0000 7f0ca5160700 1 --2- 172.21.15.43:0/3388521552 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0c8c004120 0x7f0ca01efff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:43.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:43.095+0000 7f0ca5160700 1 --2- 172.21.15.43:0/3388521552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ca00b4da0 0x7f0ca01f2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:43.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:43.095+0000 7f0ca5160700 1 -- 172.21.15.43:0/3388521552 >> 172.21.15.43:0/3388521552 conn(0x7f0ca006c170 msgr2=0x7f0ca00cfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:43.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:43.095+0000 7f0ca5160700 1 -- 172.21.15.43:0/3388521552 shutdown_connections 2023-06-28T20:08:43.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:43.095+0000 7f0ca5160700 1 -- 172.21.15.43:0/3388521552 wait complete. 2023-06-28T20:08:43.111 INFO:tasks.cephfs_test_runner:test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) ... ERROR 2023-06-28T20:08:43.112 INFO:tasks.cephfs_test_runner: 2023-06-28T20:08:43.112 INFO:tasks.cephfs_test_runner:====================================================================== 2023-06-28T20:08:43.113 INFO:tasks.cephfs_test_runner:ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-06-28T20:08:43.113 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-06-28T20:08:43.113 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-06-28T20:08:43.113 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_a7bcf4acb88d3aafb511a5f395764c81b8c88a7c/qa/tasks/cephfs/test_quota.py", line 157, in test_disable_enable_human_readable_quota_values 2023-06-28T20:08:43.113 INFO:tasks.cephfs_test_runner: self.mount_a.setfattr("./subdir", "ceph.quota.max_bytes", "1K") 2023-06-28T20:08:43.114 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_a7bcf4acb88d3aafb511a5f395764c81b8c88a7c/qa/tasks/cephfs/mount.py", line 1488, in setfattr 2023-06-28T20:08:43.114 INFO:tasks.cephfs_test_runner: self.run_shell(**kwargs) 2023-06-28T20:08:43.114 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_a7bcf4acb88d3aafb511a5f395764c81b8c88a7c/qa/tasks/cephfs/mount.py", line 780, in run_shell 2023-06-28T20:08:43.114 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-06-28T20:08:43.114 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/remote.py", line 525, in run 2023-06-28T20:08:43.115 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-06-28T20:08:43.115 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/run.py", line 455, in run 2023-06-28T20:08:43.115 INFO:tasks.cephfs_test_runner: r.wait() 2023-06-28T20:08:43.115 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/run.py", line 161, in wait 2023-06-28T20:08:43.115 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-06-28T20:08:43.116 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-06-28T20:08:43.116 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-06-28T20:08:43.116 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi043 with status 1: '(cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 1K ./subdir)' 2023-06-28T20:08:43.116 INFO:tasks.cephfs_test_runner: 2023-06-28T20:08:43.116 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-06-28T20:08:43.116 INFO:tasks.cephfs_test_runner:Ran 1 test in 50.153s 2023-06-28T20:08:43.117 INFO:tasks.cephfs_test_runner: 2023-06-28T20:08:43.117 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-06-28T20:08:43.117 INFO:tasks.cephfs_test_runner: 2023-06-28T20:08:43.117 INFO:tasks.cephfs_test_runner:====================================================================== 2023-06-28T20:08:43.118 INFO:tasks.cephfs_test_runner:ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-06-28T20:08:43.118 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-06-28T20:08:43.118 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-06-28T20:08:43.118 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_a7bcf4acb88d3aafb511a5f395764c81b8c88a7c/qa/tasks/cephfs/test_quota.py", line 157, in test_disable_enable_human_readable_quota_values 2023-06-28T20:08:43.118 INFO:tasks.cephfs_test_runner: self.mount_a.setfattr("./subdir", "ceph.quota.max_bytes", "1K") 2023-06-28T20:08:43.119 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_a7bcf4acb88d3aafb511a5f395764c81b8c88a7c/qa/tasks/cephfs/mount.py", line 1488, in setfattr 2023-06-28T20:08:43.119 INFO:tasks.cephfs_test_runner: self.run_shell(**kwargs) 2023-06-28T20:08:43.119 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_a7bcf4acb88d3aafb511a5f395764c81b8c88a7c/qa/tasks/cephfs/mount.py", line 780, in run_shell 2023-06-28T20:08:43.120 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-06-28T20:08:43.120 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/remote.py", line 525, in run 2023-06-28T20:08:43.120 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-06-28T20:08:43.120 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/run.py", line 455, in run 2023-06-28T20:08:43.120 INFO:tasks.cephfs_test_runner: r.wait() 2023-06-28T20:08:43.121 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/run.py", line 161, in wait 2023-06-28T20:08:43.121 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-06-28T20:08:43.121 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-06-28T20:08:43.121 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-06-28T20:08:43.121 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi043 with status 1: '(cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 1K ./subdir)' 2023-06-28T20:08:43.122 INFO:tasks.cephfs_test_runner: 2023-06-28T20:08:43.122 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a7bcf4acb88d3aafb511a5f395764c81b8c88a7c/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-06-28T20:08:43.207 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=f4f09624390e49ceb6a0d3bf9219ac27 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_aa9345a996190b15ab9d46b5969b5f8280bfba99/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a7bcf4acb88d3aafb511a5f395764c81b8c88a7c/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-06-28T20:08:43.209 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-06-28T20:08:43.221 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-06-28T20:08:43.304 INFO:tasks.kclient:Unmounting kernel clients... 2023-06-28T20:08:43.306 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:08:43.306 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:08:48.335 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13062 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-06-28T20:08:48.336 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2023-06-28T20:08:48.336 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:08:48.397 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=13062 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-06-28T20:08:48.398 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:08:48.398 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:08:48.419 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3625 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-06-28T20:08:48.420 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2023-06-28T20:08:48.420 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-28T20:08:48.479 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284184k,nr_inodes=4071046,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3625 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260536k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-06-28T20:08:48.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:48.480 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-06-28T20:08:48.501 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-06-28T20:08:48.501 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-06-28T20:08:48.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:48.502 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-06-28T20:08:48.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:48.592 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-06-28T20:08:48.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:48.674 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2023-06-28T20:08:48.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:48.780 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2023-06-28T20:08:48.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:48.799 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2023-06-28T20:08:48.869 INFO:teuthology.orchestra.run.smithi043.stderr:Cannot find device "ceph-brx" 2023-06-28T20:08:48.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-28T20:08:48.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:48.871 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2023-06-28T20:08:48.891 INFO:teuthology.orchestra.run.smithi178.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-06-28T20:08:48.892 INFO:teuthology.orchestra.run.smithi178.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-06-28T20:08:48.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:48.893 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-06-28T20:08:48.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:48.976 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-06-28T20:08:49.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:49.064 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link delete ceph-brx 2023-06-28T20:08:49.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:49.157 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2023-06-28T20:08:49.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-28T20:08:49.174 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link delete ceph-brx 2023-06-28T20:08:49.244 INFO:teuthology.orchestra.run.smithi178.stderr:Cannot find device "ceph-brx" 2023-06-28T20:08:49.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-28T20:08:49.246 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-06-28T20:08:49.259 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-06-28T20:08:49.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-28T20:08:49.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.388+0000 7fe9c6d5f700 1 Processor -- start 2023-06-28T20:08:49.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.388+0000 7fe9c6d5f700 1 -- start start 2023-06-28T20:08:49.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.389+0000 7fe9c6d5f700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9c013c6d0 0x7fe9c00cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:49.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.389+0000 7fe9c6d5f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe9c013ee80 0x7fe9c006bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:49.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.389+0000 7fe9c6d5f700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c015a430 con 0x7fe9c00cb1e0 2023-06-28T20:08:49.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.389+0000 7fe9c6d5f700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c0159820 con 0x7fe9c013ee80 2023-06-28T20:08:49.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.389+0000 7fe9c6d5f700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c0159b00 con 0x7fe9c013c6d0 2023-06-28T20:08:49.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.389+0000 7fe9c52fc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe9c013ee80 0x7fe9c006bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:49.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.389+0000 7fe9c52fc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe9c013ee80 0x7fe9c006bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55624/0 (socket says 172.21.15.43:55624) 2023-06-28T20:08:49.396 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.389+0000 7fe9c52fc700 1 -- 172.21.15.43:0/2509466955 learned_addr learned my addr 172.21.15.43:0/2509466955 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:49.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.389+0000 7fe9bffff700 1 --2- 172.21.15.43:0/2509466955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9c013c6d0 0x7fe9c00cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:49.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.390+0000 7fe9c52fc700 1 -- 172.21.15.43:0/2509466955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9c013c6d0 msgr2=0x7fe9c00cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:49.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.390+0000 7fe9c52fc700 1 --2- 172.21.15.43:0/2509466955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9c013c6d0 0x7fe9c00cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:49.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.390+0000 7fe9c52fc700 1 -- 172.21.15.43:0/2509466955 >> v1:172.21.15.178:6789/0 conn(0x7fe9c00cb1e0 legacy=0x7fe9c015d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:49.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.390+0000 7fe9c52fc700 1 -- 172.21.15.43:0/2509466955 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c0159de0 con 0x7fe9c013ee80 2023-06-28T20:08:49.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.390+0000 7fe9bffff700 1 --2- 172.21.15.43:0/2509466955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9c013c6d0 0x7fe9c00cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:49.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.390+0000 7fe9c52fc700 1 --2- 172.21.15.43:0/2509466955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9c013ee80 0x7fe9c006bba0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009e50 tx=0x7fe9b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f664fadae5add39 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:49.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.390+0000 7fe9bf7fe700 1 -- 172.21.15.43:0/2509466955 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b800e070 con 0x7fe9c013ee80 2023-06-28T20:08:49.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.390+0000 7fe9bf7fe700 1 -- 172.21.15.43:0/2509466955 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b800b9c0 con 0x7fe9c013ee80 2023-06-28T20:08:49.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.390+0000 7fe9bf7fe700 1 -- 172.21.15.43:0/2509466955 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b800bd70 con 0x7fe9c013ee80 2023-06-28T20:08:49.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.390+0000 7fe9c6d5f700 1 -- 172.21.15.43:0/2509466955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9c013ee80 msgr2=0x7fe9c006bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:49.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.390+0000 7fe9c6d5f700 1 --2- 172.21.15.43:0/2509466955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9c013ee80 0x7fe9c006bba0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009e50 tx=0x7fe9b800b330 comp rx=0 tx=0).stop 2023-06-28T20:08:49.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.391+0000 7fe9c6d5f700 1 -- 172.21.15.43:0/2509466955 shutdown_connections 2023-06-28T20:08:49.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.391+0000 7fe9c6d5f700 1 --2- 172.21.15.43:0/2509466955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9c013ee80 0x7fe9c006bba0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:49.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.391+0000 7fe9c6d5f700 1 --2- 172.21.15.43:0/2509466955 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9c013c6d0 0x7fe9c00cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:49.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.391+0000 7fe9c6d5f700 1 -- 172.21.15.43:0/2509466955 >> 172.21.15.43:0/2509466955 conn(0x7fe9c0069ef0 msgr2=0x7fe9c006a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:49.400 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.391+0000 7fe9c6d5f700 1 -- 172.21.15.43:0/2509466955 shutdown_connections 2023-06-28T20:08:49.401 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.391+0000 7fe9c6d5f700 1 -- 172.21.15.43:0/2509466955 wait complete. 2023-06-28T20:08:49.403 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.392+0000 7fe9c6d5f700 1 Processor -- start 2023-06-28T20:08:49.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.392+0000 7fe9c6d5f700 1 -- start start 2023-06-28T20:08:49.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.393+0000 7fe9c6d5f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9c00cb1e0 0x7fe9c01f5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:49.404 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.393+0000 7fe9c6d5f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9c013c6d0 0x7fe9c00c2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:49.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.393+0000 7fe9c6d5f700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9c013ee80 0x7fe9c00c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:49.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.393+0000 7fe9c6d5f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c00c5d40 con 0x7fe9c013c6d0 2023-06-28T20:08:49.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.393+0000 7fe9c6d5f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c013cb40 con 0x7fe9c00cb1e0 2023-06-28T20:08:49.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.393+0000 7fe9c6d5f700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c00c5fe0 con 0x7fe9c013ee80 2023-06-28T20:08:49.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.393+0000 7fe9bffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9c013c6d0 0x7fe9c00c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:49.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.393+0000 7fe9bffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9c013c6d0 0x7fe9c00c2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55628/0 (socket says 172.21.15.43:55628) 2023-06-28T20:08:49.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.393+0000 7fe9bffff700 1 -- 172.21.15.43:0/1104903387 learned_addr learned my addr 172.21.15.43:0/1104903387 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:49.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.393+0000 7fe9c4afb700 1 --2- 172.21.15.43:0/1104903387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9c00cb1e0 0x7fe9c01f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:49.407 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.393+0000 7fe9c52fc700 1 --2- 172.21.15.43:0/1104903387 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9c013ee80 0x7fe9c00c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:49.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.393+0000 7fe9bffff700 1 -- 172.21.15.43:0/1104903387 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9c013ee80 msgr2=0x7fe9c00c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:49.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.393+0000 7fe9bffff700 1 --2- 172.21.15.43:0/1104903387 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9c013ee80 0x7fe9c00c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:49.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.393+0000 7fe9bffff700 1 -- 172.21.15.43:0/1104903387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9c00cb1e0 msgr2=0x7fe9c01f5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:49.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.393+0000 7fe9bffff700 1 --2- 172.21.15.43:0/1104903387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9c00cb1e0 0x7fe9c01f5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:49.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.393+0000 7fe9bffff700 1 -- 172.21.15.43:0/1104903387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b8009b30 con 0x7fe9c013c6d0 2023-06-28T20:08:49.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.393+0000 7fe9bffff700 1 --2- 172.21.15.43:0/1104903387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9c013c6d0 0x7fe9c00c2670 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00d900 tx=0x7fe9ac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:49.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.393+0000 7fe9bd7fa700 1 -- 172.21.15.43:0/1104903387 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac004220 con 0x7fe9c013c6d0 2023-06-28T20:08:49.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.394+0000 7fe9bd7fa700 1 -- 172.21.15.43:0/1104903387 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ac004380 con 0x7fe9c013c6d0 2023-06-28T20:08:49.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.394+0000 7fe9c6d5f700 1 -- 172.21.15.43:0/1104903387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c013cde0 con 0x7fe9c013c6d0 2023-06-28T20:08:49.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.394+0000 7fe9bd7fa700 1 -- 172.21.15.43:0/1104903387 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac00b6d0 con 0x7fe9c013c6d0 2023-06-28T20:08:49.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.394+0000 7fe9c6d5f700 1 -- 172.21.15.43:0/1104903387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c00b3cf0 con 0x7fe9c013c6d0 2023-06-28T20:08:49.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.395+0000 7fe9c6d5f700 1 -- 172.21.15.43:0/1104903387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c013cf70 con 0x7fe9c013c6d0 2023-06-28T20:08:49.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.395+0000 7fe9bd7fa700 1 -- 172.21.15.43:0/1104903387 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe9ac00ba40 con 0x7fe9c013c6d0 2023-06-28T20:08:49.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.395+0000 7fe9bd7fa700 1 --2- 172.21.15.43:0/1104903387 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe9a406ace0 0x7fe9a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:49.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.396+0000 7fe9bd7fa700 1 -- 172.21.15.43:0/1104903387 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe9ac089440 con 0x7fe9c013c6d0 2023-06-28T20:08:49.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.396+0000 7fe9c4afb700 1 --2- 172.21.15.43:0/1104903387 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe9a406ace0 0x7fe9a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:49.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.396+0000 7fe9c4afb700 1 --2- 172.21.15.43:0/1104903387 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe9a406ace0 0x7fe9a406d190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4000f60 tx=0x7fe9b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:49.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.400+0000 7fe9bd7fa700 1 -- 172.21.15.43:0/1104903387 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe9ac054880 con 0x7fe9c013c6d0 2023-06-28T20:08:49.597 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.592+0000 7fe9c6d5f700 1 -- 172.21.15.43:0/1104903387 --> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe9c015a0c0 con 0x7fe9a406ace0 2023-06-28T20:08:49.598 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.595+0000 7fe9bd7fa700 1 -- 172.21.15.43:0/1104903387 <== mgr.4101 v2:172.21.15.43:6832/406144153 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72976 (secure 0 0 0) 0x7fe9b4001490 con 0x7fe9a406ace0 2023-06-28T20:08:49.599 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:49.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.597+0000 7fe9c6d5f700 1 -- 172.21.15.43:0/1104903387 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe9a406ace0 msgr2=0x7fe9a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:49.607 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.597+0000 7fe9c6d5f700 1 --2- 172.21.15.43:0/1104903387 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe9a406ace0 0x7fe9a406d190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4000f60 tx=0x7fe9b4009040 comp rx=0 tx=0).stop 2023-06-28T20:08:49.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.597+0000 7fe9c6d5f700 1 -- 172.21.15.43:0/1104903387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9c013c6d0 msgr2=0x7fe9c00c2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:49.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.597+0000 7fe9c6d5f700 1 --2- 172.21.15.43:0/1104903387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9c013c6d0 0x7fe9c00c2670 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00d900 tx=0x7fe9ac00dcc0 comp rx=0 tx=0).stop 2023-06-28T20:08:49.608 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.597+0000 7fe9c6d5f700 1 -- 172.21.15.43:0/1104903387 shutdown_connections 2023-06-28T20:08:49.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.597+0000 7fe9c6d5f700 1 --2- 172.21.15.43:0/1104903387 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe9a406ace0 0x7fe9a406d190 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:49.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.597+0000 7fe9c6d5f700 1 --2- 172.21.15.43:0/1104903387 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe9c013ee80 0x7fe9c00c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:49.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.597+0000 7fe9c6d5f700 1 --2- 172.21.15.43:0/1104903387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9c00cb1e0 0x7fe9c01f5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:49.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.597+0000 7fe9c6d5f700 1 --2- 172.21.15.43:0/1104903387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9c013c6d0 0x7fe9c00c2670 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:49.609 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.597+0000 7fe9c6d5f700 1 -- 172.21.15.43:0/1104903387 >> 172.21.15.43:0/1104903387 conn(0x7fe9c0069ef0 msgr2=0x7fe9c015a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:49.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.597+0000 7fe9c6d5f700 1 -- 172.21.15.43:0/1104903387 shutdown_connections 2023-06-28T20:08:49.610 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.597+0000 7fe9c6d5f700 1 -- 172.21.15.43:0/1104903387 wait complete. 2023-06-28T20:08:49.610 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-06-28T20:08:49.613 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2023-06-28T20:08:48.008740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220168,"kb_used_data":4552,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754552,"statfs":{"total":773094113280,"available":772868661248,"internally_reserved":0,"allocated":4661248,"data_stored":2619571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8413,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-55,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001509"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.226142+0000","last_change":"2023-06-28T20:07:15.815425+0000","last_active":"2023-06-28T20:08:42.226142+0000","last_peered":"2023-06-28T20:08:42.226142+0000","last_clean":"2023-06-28T20:08:42.226142+0000","last_became_active":"2023-06-28T20:07:15.815163+0000","last_became_peered":"2023-06-28T20:07:15.815163+0000","last_unstale":"2023-06-28T20:08:42.226142+0000","last_undegraded":"2023-06-28T20:08:42.226142+0000","last_fullsized":"2023-06-28T20:08:42.226142+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:54:07.698762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":49,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.225754+0000","last_change":"2023-06-28T20:07:15.303675+0000","last_active":"2023-06-28T20:08:42.225754+0000","last_peered":"2023-06-28T20:08:42.225754+0000","last_clean":"2023-06-28T20:08:42.225754+0000","last_became_active":"2023-06-28T20:07:15.303350+0000","last_became_peered":"2023-06-28T20:07:15.303350+0000","last_unstale":"2023-06-28T20:08:42.225754+0000","last_undegraded":"2023-06-28T20:08:42.225754+0000","last_fullsized":"2023-06-28T20:08:42.225754+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:51:41.786605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.225001+0000","last_change":"2023-06-28T20:07:15.303565+0000","last_active":"2023-06-28T20:08:42.225001+0000","last_peered":"2023-06-28T20:08:42.225001+0000","last_clean":"2023-06-28T20:08:42.225001+0000","last_became_active":"2023-06-28T20:07:15.303200+0000","last_became_peered":"2023-06-28T20:07:15.303200+0000","last_unstale":"2023-06-28T20:08:42.225001+0000","last_undegraded":"2023-06-28T20:08:42.225001+0000","last_fullsized":"2023-06-28T20:08:42.225001+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:02:44.229956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":189,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.094410+0000","last_change":"2023-06-28T20:07:15.304921+0000","last_active":"2023-06-28T20:08:42.094410+0000","last_peered":"2023-06-28T20:08:42.094410+0000","last_clean":"2023-06-28T20:08:42.094410+0000","last_became_active":"2023-06-28T20:07:15.303959+0000","last_became_peered":"2023-06-28T20:07:15.303959+0000","last_unstale":"2023-06-28T20:08:42.094410+0000","last_undegraded":"2023-06-28T20:08:42.094410+0000","last_fullsized":"2023-06-28T20:08:42.094410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:07:09.587950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":41,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.094663+0000","last_change":"2023-06-28T20:07:15.899048+0000","last_active":"2023-06-28T20:08:42.094663+0000","last_peered":"2023-06-28T20:08:42.094663+0000","last_clean":"2023-06-28T20:08:42.094663+0000","last_became_active":"2023-06-28T20:07:15.898766+0000","last_became_peered":"2023-06-28T20:07:15.898766+0000","last_unstale":"2023-06-28T20:08:42.094663+0000","last_undegraded":"2023-06-28T20:08:42.094663+0000","last_fullsized":"2023-06-28T20:08:42.094663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:32:41.079504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":39,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.094199+0000","last_change":"2023-06-28T20:07:15.899229+0000","last_active":"2023-06-28T20:08:42.094199+0000","last_peered":"2023-06-28T20:08:42.094199+0000","last_clean":"2023-06-28T20:08:42.094199+0000","last_became_active":"2023-06-28T20:07:15.898708+0000","last_became_peered":"2023-06-28T20:07:15.898708+0000","last_unstale":"2023-06-28T20:08:42.094199+0000","last_undegraded":"2023-06-28T20:08:42.094199+0000","last_fullsized":"2023-06-28T20:08:42.094199+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:11:32.901441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.227358+0000","last_change":"2023-06-28T20:07:15.898966+0000","last_active":"2023-06-28T20:08:42.227358+0000","last_peered":"2023-06-28T20:08:42.227358+0000","last_clean":"2023-06-28T20:08:42.227358+0000","last_became_active":"2023-06-28T20:07:15.898704+0000","last_became_peered":"2023-06-28T20:07:15.898704+0000","last_unstale":"2023-06-28T20:08:42.227358+0000","last_undegraded":"2023-06-28T20:08:42.227358+0000","last_fullsized":"2023-06-28T20:08:42.227358+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:19:46.078630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":53,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.095830+0000","last_change":"2023-06-28T20:07:15.899207+0000","last_active":"2023-06-28T20:08:42.095830+0000","last_peered":"2023-06-28T20:08:42.095830+0000","last_clean":"2023-06-28T20:08:42.095830+0000","last_became_active":"2023-06-28T20:07:15.898908+0000","last_became_peered":"2023-06-28T20:07:15.898908+0000","last_unstale":"2023-06-28T20:08:42.095830+0000","last_undegraded":"2023-06-28T20:08:42.095830+0000","last_fullsized":"2023-06-28T20:08:42.095830+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:58:47.410109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":53,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.095885+0000","last_change":"2023-06-28T20:07:15.303873+0000","last_active":"2023-06-28T20:08:42.095885+0000","last_peered":"2023-06-28T20:08:42.095885+0000","last_clean":"2023-06-28T20:08:42.095885+0000","last_became_active":"2023-06-28T20:07:15.303457+0000","last_became_peered":"2023-06-28T20:07:15.303457+0000","last_unstale":"2023-06-28T20:08:42.095885+0000","last_undegraded":"2023-06-28T20:08:42.095885+0000","last_fullsized":"2023-06-28T20:08:42.095885+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:20:41.981562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":1036288,"data_stored":774050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61899999999999999}]},{"osd":1,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.42599999999999999}]},{"osd":2,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.57399999999999995}]},{"osd":3,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.64900000000000002}]},{"osd":4,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.47199999999999998}]},{"osd":5,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.54300000000000004}]},{"osd":6,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.502},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.66300000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":183682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56200000000000006},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":9.0069999999999997,"5min":9.0069999999999997,"15min":9.0069999999999997},"last":0.47599999999999998}]},{"osd":1,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":3.036,"5min":3.036,"15min":3.036},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.66900000000000004}]},{"osd":2,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.76800000000000002}]},{"osd":3,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":9.0459999999999994,"5min":9.0459999999999994,"15min":9.0459999999999994},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.68500000000000005}]},{"osd":4,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.64100000000000001}]},{"osd":5,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.72399999999999998}]},{"osd":7,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.75},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.70399999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":1007616,"data_stored":761866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.253},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.29799999999999999}]},{"osd":2,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.35799999999999998}]},{"osd":3,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.34399999999999997}]},{"osd":4,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56399999999999995}]},{"osd":5,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.55100000000000005}]},{"osd":6,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.501},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.58899999999999997}]},{"osd":7,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.46300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":438272,"data_stored":182721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.495}]},{"osd":2,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.442,"5min":0.442,"15min":0.442},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.41699999999999998}]},{"osd":3,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.51600000000000001}]},{"osd":4,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.60699999999999998}]},{"osd":5,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.64700000000000002}]},{"osd":6,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.80700000000000005}]},{"osd":7,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.755}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":183682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":16.044,"5min":16.044,"15min":16.044},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.44900000000000001}]},{"osd":1,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":11.028,"5min":11.028,"15min":11.028},"last":0.32700000000000001},{"interface":"front","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":16.077000000000002,"5min":16.077000000000002,"15min":16.077000000000002},"last":0.40200000000000002}]},{"osd":3,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.28100000000000003},{"interface":"front","average":{"1min":1.319,"5min":1.319,"15min":1.319},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":16.004000000000001,"5min":16.004000000000001,"15min":16.004000000000001},"last":0.42899999999999999}]},{"osd":4,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":9.327,"5min":9.327,"15min":9.327},"last":0.621},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":10.964,"5min":10.964,"15min":10.964},"last":0.69199999999999995}]},{"osd":5,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":10.923,"5min":10.923,"15min":10.923},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.72299999999999998}]},{"osd":6,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":9.282,"5min":9.282,"15min":9.282},"last":0.56000000000000005}]},{"osd":7,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":10.996,"5min":10.996,"15min":10.996},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.60199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":183682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.307},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":5.0250000000000004,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":0.252}]},{"osd":1,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":5.2309999999999999,"5min":5.2309999999999999,"15min":5.2309999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":5.3250000000000002,"5min":5.3250000000000002,"15min":5.3250000000000002},"last":0.27400000000000002}]},{"osd":2,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":15.927,"5min":15.927,"15min":15.927},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.29399999999999998}]},{"osd":4,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.40600000000000003}]},{"osd":5,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":4.9939999999999998,"5min":4.9939999999999998,"15min":4.9939999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.42899999999999999}]},{"osd":6,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":4.9619999999999997,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.443}]},{"osd":7,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":4.9210000000000003,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.45900000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":424,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":434176,"data_stored":178368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":9.1809999999999992,"5min":9.1809999999999992,"15min":9.1809999999999992},"last":0.33600000000000002}]},{"osd":1,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.435},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.45400000000000001}]},{"osd":2,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":9.0700000000000003,"5min":9.0700000000000003,"15min":9.0700000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.48099999999999998}]},{"osd":3,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.38},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.52000000000000002}]},{"osd":5,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.57899999999999996}]},{"osd":6,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.622},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.54300000000000004}]},{"osd":7,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.64700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":417792,"data_stored":171520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.5}]},{"osd":1,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.48099999999999998}]},{"osd":2,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.435},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.58499999999999996}]},{"osd":3,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":5.6139999999999999,"5min":5.6139999999999999,"15min":5.6139999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":5.6500000000000004,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.60099999999999998}]},{"osd":4,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.65500000000000003}]},{"osd":6,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.46500000000000002}]},{"osd":7,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.63800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-28T20:08:49.616 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-28T20:08:49.745 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.738+0000 7fd9c09e8700 1 Processor -- start 2023-06-28T20:08:49.746 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.738+0000 7fd9c09e8700 1 -- start start 2023-06-28T20:08:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.739+0000 7fd9c09e8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9bc15d9f0 0x7fd9bc06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.739+0000 7fd9c09e8700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9bc071170 con 0x7fd9bc070cb0 2023-06-28T20:08:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.739+0000 7fd9c09e8700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9bc071450 con 0x7fd9bc06a600 2023-06-28T20:08:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.739+0000 7fd9c09e8700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc15ddc0 con 0x7fd9bc15d9f0 2023-06-28T20:08:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.739+0000 7fd9b9d9b700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd9bc070cb0 0x7fd9bc069ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44824/0 (socket says 172.21.15.43:44824) 2023-06-28T20:08:49.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.739+0000 7fd9b9d9b700 1 -- 172.21.15.43:0/2380982503 learned_addr learned my addr 172.21.15.43:0/2380982503 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:49.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9ba59c700 1 --2- 172.21.15.43:0/2380982503 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9bc15d9f0 0x7fd9bc06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:49.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9b959a700 1 -- 172.21.15.43:0/2380982503 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1045209542 0 0) 0x7fd9bc071170 con 0x7fd9bc070cb0 2023-06-28T20:08:49.748 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9b959a700 1 -- 172.21.15.43:0/2380982503 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9a40035b0 con 0x7fd9bc070cb0 2023-06-28T20:08:49.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9b959a700 1 -- 172.21.15.43:0/2380982503 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 62392869 0 0) 0x7fd9bc071450 con 0x7fd9bc06a600 2023-06-28T20:08:49.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9b959a700 1 -- 172.21.15.43:0/2380982503 --> v1:172.21.15.178:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9bc071170 con 0x7fd9bc06a600 2023-06-28T20:08:49.749 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9b959a700 1 -- 172.21.15.43:0/2380982503 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2282333901 0 0) 0x7fd9a40035b0 con 0x7fd9bc070cb0 2023-06-28T20:08:49.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9b959a700 1 -- 172.21.15.43:0/2380982503 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9bc15d9f0 msgr2=0x7fd9bc06d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:49.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9b959a700 1 --2- 172.21.15.43:0/2380982503 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9bc15d9f0 0x7fd9bc06d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:49.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9b959a700 1 -- 172.21.15.43:0/2380982503 >> v1:172.21.15.178:6789/0 conn(0x7fd9bc06a600 legacy=0x7fd9bc0cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:49.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9b959a700 1 -- 172.21.15.43:0/2380982503 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9bc0cb240 con 0x7fd9bc070cb0 2023-06-28T20:08:49.750 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9b959a700 1 -- 172.21.15.43:0/2380982503 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7fd9ac001670 con 0x7fd9bc070cb0 2023-06-28T20:08:49.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9b959a700 1 -- 172.21.15.43:0/2380982503 >> v1:172.21.15.43:6789/0 conn(0x7fd9bc070cb0 legacy=0x7fd9bc069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:49.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9b959a700 1 --2- 172.21.15.43:0/2380982503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9a4003bc0 0x7fd9a4003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:49.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9b959a700 1 --2- 172.21.15.43:0/2380982503 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9bc15d9f0 0x7fd9a4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:49.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9b959a700 1 --2- 172.21.15.43:0/2380982503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a4004d90 0x7fd9a4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:49.751 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9b959a700 1 -- 172.21.15.43:0/2380982503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc15ddc0 con 0x7fd9a4004d90 2023-06-28T20:08:49.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9b959a700 1 -- 172.21.15.43:0/2380982503 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a400bb10 con 0x7fd9a4003bc0 2023-06-28T20:08:49.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9b959a700 1 -- 172.21.15.43:0/2380982503 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a400bcd0 con 0x7fd9bc15d9f0 2023-06-28T20:08:49.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9bad9d700 1 --2- 172.21.15.43:0/2380982503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a4004d90 0x7fd9a4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:49.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9ba59c700 1 --2- 172.21.15.43:0/2380982503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9a4003bc0 0x7fd9a4003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:49.752 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9b9d9b700 1 --2- 172.21.15.43:0/2380982503 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9bc15d9f0 0x7fd9a4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:49.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9bad9d700 1 -- 172.21.15.43:0/2380982503 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9bc15d9f0 msgr2=0x7fd9a4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:49.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9bad9d700 1 --2- 172.21.15.43:0/2380982503 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9bc15d9f0 0x7fd9a4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:49.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9bad9d700 1 -- 172.21.15.43:0/2380982503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9a4003bc0 msgr2=0x7fd9a4003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:49.753 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9bad9d700 1 --2- 172.21.15.43:0/2380982503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9a4003bc0 0x7fd9a4003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:49.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.740+0000 7fd9bad9d700 1 -- 172.21.15.43:0/2380982503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a40035b0 con 0x7fd9a4004d90 2023-06-28T20:08:49.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.741+0000 7fd9bad9d700 1 --2- 172.21.15.43:0/2380982503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a4004d90 0x7fd9a4005200 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0af640 tx=0x7fd9a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b90417cc95d7f5 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:49.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.741+0000 7fd9b959a700 1 -- 172.21.15.43:0/2380982503 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4001f30 con 0x7fd9a4004d90 2023-06-28T20:08:49.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.741+0000 7fd9b959a700 1 -- 172.21.15.43:0/2380982503 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b400ce30 con 0x7fd9a4004d90 2023-06-28T20:08:49.754 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.741+0000 7fd9b959a700 1 -- 172.21.15.43:0/2380982503 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4003c40 con 0x7fd9a4004d90 2023-06-28T20:08:49.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.741+0000 7fd9c09e8700 1 -- 172.21.15.43:0/2380982503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a4004d90 msgr2=0x7fd9a4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:49.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.741+0000 7fd9c09e8700 1 --2- 172.21.15.43:0/2380982503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9a4004d90 0x7fd9a4005200 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0af640 tx=0x7fd9a4005880 comp rx=0 tx=0).stop 2023-06-28T20:08:49.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.741+0000 7fd9bad9d700 1 -- 172.21.15.43:0/2380982503 reap_dead start 2023-06-28T20:08:49.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.741+0000 7fd9c09e8700 1 -- 172.21.15.43:0/2380982503 shutdown_connections 2023-06-28T20:08:49.755 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.741+0000 7fd9c09e8700 1 -- 172.21.15.43:0/2380982503 >> 172.21.15.43:0/2380982503 conn(0x7fd9bc0b1720 msgr2=0x7fd9bc0b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:49.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.741+0000 7fd9c09e8700 1 -- 172.21.15.43:0/2380982503 shutdown_connections 2023-06-28T20:08:49.756 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.741+0000 7fd9c09e8700 1 -- 172.21.15.43:0/2380982503 wait complete. 2023-06-28T20:08:49.762 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.743+0000 7fd9c09e8700 1 Processor -- start 2023-06-28T20:08:49.764 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.743+0000 7fd9c09e8700 1 -- start start 2023-06-28T20:08:49.765 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.744+0000 7fd9c09e8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9a4004d90 0x7fd9bc1f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:49.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.744+0000 7fd9c09e8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9bc0c9df0 0x7fd9bc15c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:49.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.744+0000 7fd9c09e8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9bc1efaa0 0x7fd9bc0b3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:49.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.744+0000 7fd9c09e8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc1f0a70 con 0x7fd9bc1efaa0 2023-06-28T20:08:49.767 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.744+0000 7fd9c09e8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc1f3f80 con 0x7fd9bc0c9df0 2023-06-28T20:08:49.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.744+0000 7fd9c09e8700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc1f04b0 con 0x7fd9a4004d90 2023-06-28T20:08:49.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.744+0000 7fd9bad9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9bc1efaa0 0x7fd9bc0b3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:49.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.744+0000 7fd9bad9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9bc1efaa0 0x7fd9bc0b3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55650/0 (socket says 172.21.15.43:55650) 2023-06-28T20:08:49.768 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.744+0000 7fd9bad9d700 1 -- 172.21.15.43:0/3788157817 learned_addr learned my addr 172.21.15.43:0/3788157817 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:49.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.744+0000 7fd9b9d9b700 1 --2- 172.21.15.43:0/3788157817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9bc0c9df0 0x7fd9bc15c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:49.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.744+0000 7fd9ba59c700 1 --2- 172.21.15.43:0/3788157817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9a4004d90 0x7fd9bc1f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:49.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.745+0000 7fd9bad9d700 1 -- 172.21.15.43:0/3788157817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9a4004d90 msgr2=0x7fd9bc1f3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:49.769 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.745+0000 7fd9bad9d700 1 --2- 172.21.15.43:0/3788157817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9a4004d90 0x7fd9bc1f3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:49.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.745+0000 7fd9bad9d700 1 -- 172.21.15.43:0/3788157817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9bc0c9df0 msgr2=0x7fd9bc15c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:49.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.745+0000 7fd9bad9d700 1 --2- 172.21.15.43:0/3788157817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9bc0c9df0 0x7fd9bc15c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:49.770 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.745+0000 7fd9bad9d700 1 -- 172.21.15.43:0/3788157817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9bc1f0790 con 0x7fd9bc1efaa0 2023-06-28T20:08:49.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.745+0000 7fd9b9d9b700 1 --2- 172.21.15.43:0/3788157817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9bc0c9df0 0x7fd9bc15c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:49.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.745+0000 7fd9bad9d700 1 --2- 172.21.15.43:0/3788157817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9bc1efaa0 0x7fd9bc0b3c00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4004000 tx=0x7fd9b4001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:49.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.745+0000 7fd9aaffd700 1 -- 172.21.15.43:0/3788157817 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b400cc30 con 0x7fd9bc1efaa0 2023-06-28T20:08:49.771 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.745+0000 7fd9aaffd700 1 -- 172.21.15.43:0/3788157817 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b40050d0 con 0x7fd9bc1efaa0 2023-06-28T20:08:49.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.745+0000 7fd9c09e8700 1 -- 172.21.15.43:0/3788157817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9bc1f3240 con 0x7fd9bc1efaa0 2023-06-28T20:08:49.772 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.745+0000 7fd9aaffd700 1 -- 172.21.15.43:0/3788157817 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4005480 con 0x7fd9bc1efaa0 2023-06-28T20:08:49.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.745+0000 7fd9c09e8700 1 -- 172.21.15.43:0/3788157817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9bc15c730 con 0x7fd9bc1efaa0 2023-06-28T20:08:49.773 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.746+0000 7fd9c09e8700 1 -- 172.21.15.43:0/3788157817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9bc1f30a0 con 0x7fd9bc1efaa0 2023-06-28T20:08:49.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.746+0000 7fd9aaffd700 1 -- 172.21.15.43:0/3788157817 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd9b4019070 con 0x7fd9bc1efaa0 2023-06-28T20:08:49.774 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.747+0000 7fd9aaffd700 1 --2- 172.21.15.43:0/3788157817 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd99c06ace0 0x7fd99c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:49.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.747+0000 7fd9aaffd700 1 -- 172.21.15.43:0/3788157817 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd9b4089b40 con 0x7fd9bc1efaa0 2023-06-28T20:08:49.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.747+0000 7fd9ba59c700 1 --2- 172.21.15.43:0/3788157817 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd99c06ace0 0x7fd99c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:49.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.747+0000 7fd9ba59c700 1 --2- 172.21.15.43:0/3788157817 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd99c06ace0 0x7fd99c06d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0ccef0 tx=0x7fd9b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:49.776 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.751+0000 7fd9aaffd700 1 -- 172.21.15.43:0/3788157817 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd9bc1f30a0 con 0x7fd9bc1efaa0 2023-06-28T20:08:49.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.945+0000 7fd9c09e8700 1 -- 172.21.15.43:0/3788157817 --> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd9bc0cb520 con 0x7fd99c06ace0 2023-06-28T20:08:49.950 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.948+0000 7fd9aaffd700 1 -- 172.21.15.43:0/3788157817 <== mgr.4101 v2:172.21.15.43:6832/406144153 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72976 (secure 0 0 0) 0x7fd9b0009810 con 0x7fd99c06ace0 2023-06-28T20:08:49.952 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:49.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.950+0000 7fd9c09e8700 1 -- 172.21.15.43:0/3788157817 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd99c06ace0 msgr2=0x7fd99c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:49.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.950+0000 7fd9c09e8700 1 --2- 172.21.15.43:0/3788157817 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd99c06ace0 0x7fd99c06d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0ccef0 tx=0x7fd9b0008040 comp rx=0 tx=0).stop 2023-06-28T20:08:49.953 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.950+0000 7fd9c09e8700 1 -- 172.21.15.43:0/3788157817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9bc1efaa0 msgr2=0x7fd9bc0b3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:49.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.950+0000 7fd9c09e8700 1 --2- 172.21.15.43:0/3788157817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9bc1efaa0 0x7fd9bc0b3c00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4004000 tx=0x7fd9b4001650 comp rx=0 tx=0).stop 2023-06-28T20:08:49.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.950+0000 7fd9c09e8700 1 -- 172.21.15.43:0/3788157817 shutdown_connections 2023-06-28T20:08:49.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.950+0000 7fd9c09e8700 1 --2- 172.21.15.43:0/3788157817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9bc1efaa0 0x7fd9bc0b3c00 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:49.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.950+0000 7fd9c09e8700 1 --2- 172.21.15.43:0/3788157817 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fd99c06ace0 0x7fd99c06d190 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:49.954 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.950+0000 7fd9c09e8700 1 --2- 172.21.15.43:0/3788157817 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fd9a4004d90 0x7fd9bc1f3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:49.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.950+0000 7fd9c09e8700 1 --2- 172.21.15.43:0/3788157817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9bc0c9df0 0x7fd9bc15c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:49.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.950+0000 7fd9c09e8700 1 -- 172.21.15.43:0/3788157817 >> 172.21.15.43:0/3788157817 conn(0x7fd9bc0b1720 msgr2=0x7fd9bc0b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:49.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.950+0000 7fd9c09e8700 1 -- 172.21.15.43:0/3788157817 shutdown_connections 2023-06-28T20:08:49.955 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:49.950+0000 7fd9c09e8700 1 -- 172.21.15.43:0/3788157817 wait complete. 2023-06-28T20:08:49.956 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-06-28T20:08:49.965 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2023-06-28T20:08:48.008740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220168,"kb_used_data":4552,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754552,"statfs":{"total":773094113280,"available":772868661248,"internally_reserved":0,"allocated":4661248,"data_stored":2619571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8413,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-55,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001509"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.226142+0000","last_change":"2023-06-28T20:07:15.815425+0000","last_active":"2023-06-28T20:08:42.226142+0000","last_peered":"2023-06-28T20:08:42.226142+0000","last_clean":"2023-06-28T20:08:42.226142+0000","last_became_active":"2023-06-28T20:07:15.815163+0000","last_became_peered":"2023-06-28T20:07:15.815163+0000","last_unstale":"2023-06-28T20:08:42.226142+0000","last_undegraded":"2023-06-28T20:08:42.226142+0000","last_fullsized":"2023-06-28T20:08:42.226142+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:54:07.698762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":49,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.225754+0000","last_change":"2023-06-28T20:07:15.303675+0000","last_active":"2023-06-28T20:08:42.225754+0000","last_peered":"2023-06-28T20:08:42.225754+0000","last_clean":"2023-06-28T20:08:42.225754+0000","last_became_active":"2023-06-28T20:07:15.303350+0000","last_became_peered":"2023-06-28T20:07:15.303350+0000","last_unstale":"2023-06-28T20:08:42.225754+0000","last_undegraded":"2023-06-28T20:08:42.225754+0000","last_fullsized":"2023-06-28T20:08:42.225754+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:51:41.786605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.225001+0000","last_change":"2023-06-28T20:07:15.303565+0000","last_active":"2023-06-28T20:08:42.225001+0000","last_peered":"2023-06-28T20:08:42.225001+0000","last_clean":"2023-06-28T20:08:42.225001+0000","last_became_active":"2023-06-28T20:07:15.303200+0000","last_became_peered":"2023-06-28T20:07:15.303200+0000","last_unstale":"2023-06-28T20:08:42.225001+0000","last_undegraded":"2023-06-28T20:08:42.225001+0000","last_fullsized":"2023-06-28T20:08:42.225001+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:02:44.229956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":189,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.094410+0000","last_change":"2023-06-28T20:07:15.304921+0000","last_active":"2023-06-28T20:08:42.094410+0000","last_peered":"2023-06-28T20:08:42.094410+0000","last_clean":"2023-06-28T20:08:42.094410+0000","last_became_active":"2023-06-28T20:07:15.303959+0000","last_became_peered":"2023-06-28T20:07:15.303959+0000","last_unstale":"2023-06-28T20:08:42.094410+0000","last_undegraded":"2023-06-28T20:08:42.094410+0000","last_fullsized":"2023-06-28T20:08:42.094410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:07:09.587950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":41,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.094663+0000","last_change":"2023-06-28T20:07:15.899048+0000","last_active":"2023-06-28T20:08:42.094663+0000","last_peered":"2023-06-28T20:08:42.094663+0000","last_clean":"2023-06-28T20:08:42.094663+0000","last_became_active":"2023-06-28T20:07:15.898766+0000","last_became_peered":"2023-06-28T20:07:15.898766+0000","last_unstale":"2023-06-28T20:08:42.094663+0000","last_undegraded":"2023-06-28T20:08:42.094663+0000","last_fullsized":"2023-06-28T20:08:42.094663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:32:41.079504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":39,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.094199+0000","last_change":"2023-06-28T20:07:15.899229+0000","last_active":"2023-06-28T20:08:42.094199+0000","last_peered":"2023-06-28T20:08:42.094199+0000","last_clean":"2023-06-28T20:08:42.094199+0000","last_became_active":"2023-06-28T20:07:15.898708+0000","last_became_peered":"2023-06-28T20:07:15.898708+0000","last_unstale":"2023-06-28T20:08:42.094199+0000","last_undegraded":"2023-06-28T20:08:42.094199+0000","last_fullsized":"2023-06-28T20:08:42.094199+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:11:32.901441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.227358+0000","last_change":"2023-06-28T20:07:15.898966+0000","last_active":"2023-06-28T20:08:42.227358+0000","last_peered":"2023-06-28T20:08:42.227358+0000","last_clean":"2023-06-28T20:08:42.227358+0000","last_became_active":"2023-06-28T20:07:15.898704+0000","last_became_peered":"2023-06-28T20:07:15.898704+0000","last_unstale":"2023-06-28T20:08:42.227358+0000","last_undegraded":"2023-06-28T20:08:42.227358+0000","last_fullsized":"2023-06-28T20:08:42.227358+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:19:46.078630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":53,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.095830+0000","last_change":"2023-06-28T20:07:15.899207+0000","last_active":"2023-06-28T20:08:42.095830+0000","last_peered":"2023-06-28T20:08:42.095830+0000","last_clean":"2023-06-28T20:08:42.095830+0000","last_became_active":"2023-06-28T20:07:15.898908+0000","last_became_peered":"2023-06-28T20:07:15.898908+0000","last_unstale":"2023-06-28T20:08:42.095830+0000","last_undegraded":"2023-06-28T20:08:42.095830+0000","last_fullsized":"2023-06-28T20:08:42.095830+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:58:47.410109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":53,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.095885+0000","last_change":"2023-06-28T20:07:15.303873+0000","last_active":"2023-06-28T20:08:42.095885+0000","last_peered":"2023-06-28T20:08:42.095885+0000","last_clean":"2023-06-28T20:08:42.095885+0000","last_became_active":"2023-06-28T20:07:15.303457+0000","last_became_peered":"2023-06-28T20:07:15.303457+0000","last_unstale":"2023-06-28T20:08:42.095885+0000","last_undegraded":"2023-06-28T20:08:42.095885+0000","last_fullsized":"2023-06-28T20:08:42.095885+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:20:41.981562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":1036288,"data_stored":774050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61899999999999999}]},{"osd":1,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.42599999999999999}]},{"osd":2,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.57399999999999995}]},{"osd":3,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.64900000000000002}]},{"osd":4,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.47199999999999998}]},{"osd":5,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.54300000000000004}]},{"osd":6,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.502},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.66300000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":183682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56200000000000006},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":9.0069999999999997,"5min":9.0069999999999997,"15min":9.0069999999999997},"last":0.47599999999999998}]},{"osd":1,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":3.036,"5min":3.036,"15min":3.036},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.66900000000000004}]},{"osd":2,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.76800000000000002}]},{"osd":3,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":9.0459999999999994,"5min":9.0459999999999994,"15min":9.0459999999999994},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.68500000000000005}]},{"osd":4,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.64100000000000001}]},{"osd":5,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.72399999999999998}]},{"osd":7,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.75},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.70399999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":1007616,"data_stored":761866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.253},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.29799999999999999}]},{"osd":2,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.35799999999999998}]},{"osd":3,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.34399999999999997}]},{"osd":4,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.56399999999999995}]},{"osd":5,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.55100000000000005}]},{"osd":6,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.501},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.58899999999999997}]},{"osd":7,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.46300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":438272,"data_stored":182721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.495}]},{"osd":2,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.442,"5min":0.442,"15min":0.442},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.41699999999999998}]},{"osd":3,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.51600000000000001}]},{"osd":4,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.60699999999999998}]},{"osd":5,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.64700000000000002}]},{"osd":6,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.80700000000000005}]},{"osd":7,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.755}]}]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":183682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":16.044,"5min":16.044,"15min":16.044},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.44900000000000001}]},{"osd":1,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":11.028,"5min":11.028,"15min":11.028},"last":0.32700000000000001},{"interface":"front","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":16.077000000000002,"5min":16.077000000000002,"15min":16.077000000000002},"last":0.40200000000000002}]},{"osd":3,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.28100000000000003},{"interface":"front","average":{"1min":1.319,"5min":1.319,"15min":1.319},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":16.004000000000001,"5min":16.004000000000001,"15min":16.004000000000001},"last":0.42899999999999999}]},{"osd":4,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":9.327,"5min":9.327,"15min":9.327},"last":0.621},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":10.964,"5min":10.964,"15min":10.964},"last":0.69199999999999995}]},{"osd":5,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":10.923,"5min":10.923,"15min":10.923},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.72299999999999998}]},{"osd":6,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":9.282,"5min":9.282,"15min":9.282},"last":0.56000000000000005}]},{"osd":7,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":10.996,"5min":10.996,"15min":10.996},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.60199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":183682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.307},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":5.0250000000000004,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":0.252}]},{"osd":1,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":5.2309999999999999,"5min":5.2309999999999999,"15min":5.2309999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":5.3250000000000002,"5min":5.3250000000000002,"15min":5.3250000000000002},"last":0.27400000000000002}]},{"osd":2,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":15.927,"5min":15.927,"15min":15.927},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.29399999999999998}]},{"osd":4,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.40600000000000003}]},{"osd":5,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":4.9939999999999998,"5min":4.9939999999999998,"15min":4.9939999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.42899999999999999}]},{"osd":6,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":4.9619999999999997,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.443}]},{"osd":7,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":4.9210000000000003,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.45900000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":424,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":434176,"data_stored":178368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":9.1809999999999992,"5min":9.1809999999999992,"15min":9.1809999999999992},"last":0.33600000000000002}]},{"osd":1,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.435},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.45400000000000001}]},{"osd":2,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":9.0700000000000003,"5min":9.0700000000000003,"15min":9.0700000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.48099999999999998}]},{"osd":3,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.38},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.52000000000000002}]},{"osd":5,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.57899999999999996}]},{"osd":6,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.622},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.54300000000000004}]},{"osd":7,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.64700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":408,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":417792,"data_stored":171520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.5}]},{"osd":1,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.48099999999999998}]},{"osd":2,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.435},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.58499999999999996}]},{"osd":3,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":5.6139999999999999,"5min":5.6139999999999999,"15min":5.6139999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":5.6500000000000004,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.60099999999999998}]},{"osd":4,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.65500000000000003}]},{"osd":6,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.46500000000000002}]},{"osd":7,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.63800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-28T20:08:49.969 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-06-28T20:08:49.970 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-28T20:08:50.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.092+0000 7f556039a700 1 Processor -- start 2023-06-28T20:08:50.095 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.092+0000 7f556039a700 1 -- start start 2023-06-28T20:08:50.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.093+0000 7f556039a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55580d10f0 0x7f55580ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.093+0000 7f556039a700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f55580cbab0 0x7f55580b0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.093+0000 7f556039a700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555815ab50 con 0x7f55580cedf0 2023-06-28T20:08:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.093+0000 7f556039a700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55580cbf30 con 0x7f55580cbab0 2023-06-28T20:08:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.093+0000 7f556039a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55580cf260 con 0x7f55580d10f0 2023-06-28T20:08:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.093+0000 7f555e937700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f55580cbab0 0x7f55580b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.093+0000 7f555e136700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55580d10f0 0x7f55580ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.093+0000 7f555e937700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f55580cbab0 0x7f55580b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:41722/0 (socket says 172.21.15.43:41722) 2023-06-28T20:08:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.093+0000 7f555e937700 1 -- 172.21.15.43:0/4134959613 learned_addr learned my addr 172.21.15.43:0/4134959613 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.093+0000 7f555e937700 1 -- 172.21.15.43:0/4134959613 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55580d10f0 msgr2=0x7f55580ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.093+0000 7f555e937700 1 --2- 172.21.15.43:0/4134959613 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55580d10f0 0x7f55580ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.093+0000 7f555e937700 1 -- 172.21.15.43:0/4134959613 >> v1:172.21.15.178:6789/0 conn(0x7f55580cedf0 legacy=0x7f55580cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.093+0000 7f555e937700 1 -- 172.21.15.43:0/4134959613 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55580cd1b0 con 0x7f55580cbab0 2023-06-28T20:08:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.094+0000 7f555e937700 1 --2- 172.21.15.43:0/4134959613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55580cbab0 0x7f55580b0600 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5550009e80 tx=0x7f555000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48f2392d89482113 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.094+0000 7f555d134700 1 -- 172.21.15.43:0/4134959613 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555000e070 con 0x7f55580cbab0 2023-06-28T20:08:50.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.094+0000 7f555d134700 1 -- 172.21.15.43:0/4134959613 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555000b8d0 con 0x7f55580cbab0 2023-06-28T20:08:50.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.094+0000 7f555d134700 1 -- 172.21.15.43:0/4134959613 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555000bc80 con 0x7f55580cbab0 2023-06-28T20:08:50.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.094+0000 7f556039a700 1 -- 172.21.15.43:0/4134959613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55580cbab0 msgr2=0x7f55580b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:50.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.094+0000 7f556039a700 1 --2- 172.21.15.43:0/4134959613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55580cbab0 0x7f55580b0600 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5550009e80 tx=0x7f555000b330 comp rx=0 tx=0).stop 2023-06-28T20:08:50.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.094+0000 7f556039a700 1 -- 172.21.15.43:0/4134959613 shutdown_connections 2023-06-28T20:08:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.094+0000 7f556039a700 1 --2- 172.21.15.43:0/4134959613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55580cbab0 0x7f55580b0600 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.094+0000 7f556039a700 1 --2- 172.21.15.43:0/4134959613 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55580d10f0 0x7f55580ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.094+0000 7f556039a700 1 -- 172.21.15.43:0/4134959613 >> 172.21.15.43:0/4134959613 conn(0x7f55580664b0 msgr2=0x7f555809a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.094+0000 7f556039a700 1 -- 172.21.15.43:0/4134959613 shutdown_connections 2023-06-28T20:08:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.095+0000 7f556039a700 1 -- 172.21.15.43:0/4134959613 wait complete. 2023-06-28T20:08:50.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.096+0000 7f556039a700 1 Processor -- start 2023-06-28T20:08:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.096+0000 7f556039a700 1 -- start start 2023-06-28T20:08:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.097+0000 7f556039a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55580cbab0 0x7f55580ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.097+0000 7f556039a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55580cedf0 0x7f5558158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.097+0000 7f556039a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55580d10f0 0x7f55580c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:50.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.097+0000 7f556039a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55580b53d0 con 0x7f55580d10f0 2023-06-28T20:08:50.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.097+0000 7f556039a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55580c82f0 con 0x7f55580cedf0 2023-06-28T20:08:50.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.097+0000 7f556039a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55580b5080 con 0x7f55580cbab0 2023-06-28T20:08:50.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.097+0000 7f555e937700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55580d10f0 0x7f55580c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:50.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.097+0000 7f555e937700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55580d10f0 0x7f55580c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55664/0 (socket says 172.21.15.43:55664) 2023-06-28T20:08:50.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.097+0000 7f555e937700 1 -- 172.21.15.43:0/2910007101 learned_addr learned my addr 172.21.15.43:0/2910007101 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:50.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.097+0000 7f555d935700 1 --2- 172.21.15.43:0/2910007101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55580cedf0 0x7f5558158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:50.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.097+0000 7f555e136700 1 --2- 172.21.15.43:0/2910007101 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55580cbab0 0x7f55580ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:50.107 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.098+0000 7f555e937700 1 -- 172.21.15.43:0/2910007101 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55580cbab0 msgr2=0x7f55580ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:50.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.098+0000 7f555e937700 1 --2- 172.21.15.43:0/2910007101 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55580cbab0 0x7f55580ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:50.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.098+0000 7f555e937700 1 -- 172.21.15.43:0/2910007101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55580cedf0 msgr2=0x7f5558158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:50.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.098+0000 7f555e937700 1 --2- 172.21.15.43:0/2910007101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55580cedf0 0x7f5558158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:50.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.098+0000 7f555e937700 1 -- 172.21.15.43:0/2910007101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5550009b30 con 0x7f55580d10f0 2023-06-28T20:08:50.108 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.098+0000 7f555d935700 1 --2- 172.21.15.43:0/2910007101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55580cedf0 0x7f5558158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:50.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.098+0000 7f555e937700 1 --2- 172.21.15.43:0/2910007101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55580d10f0 0x7f55580c7db0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f555000b680 tx=0x7f5550004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:50.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.098+0000 7f554effd700 1 -- 172.21.15.43:0/2910007101 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55500099b0 con 0x7f55580d10f0 2023-06-28T20:08:50.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.098+0000 7f554effd700 1 -- 172.21.15.43:0/2910007101 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55500092e0 con 0x7f55580d10f0 2023-06-28T20:08:50.109 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.098+0000 7f556039a700 1 -- 172.21.15.43:0/2910007101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55580b4e70 con 0x7f55580d10f0 2023-06-28T20:08:50.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.098+0000 7f554effd700 1 -- 172.21.15.43:0/2910007101 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555001c640 con 0x7f55580d10f0 2023-06-28T20:08:50.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.098+0000 7f556039a700 1 -- 172.21.15.43:0/2910007101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55580cad50 con 0x7f55580d10f0 2023-06-28T20:08:50.110 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.099+0000 7f556039a700 1 -- 172.21.15.43:0/2910007101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55580b5510 con 0x7f55580d10f0 2023-06-28T20:08:50.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.100+0000 7f554effd700 1 -- 172.21.15.43:0/2910007101 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f555001c7a0 con 0x7f55580d10f0 2023-06-28T20:08:50.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.100+0000 7f554effd700 1 --2- 172.21.15.43:0/2910007101 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f553c06ace0 0x7f553c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:50.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.100+0000 7f554effd700 1 -- 172.21.15.43:0/2910007101 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f555008aad0 con 0x7f55580d10f0 2023-06-28T20:08:50.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.100+0000 7f555e136700 1 --2- 172.21.15.43:0/2910007101 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f553c06ace0 0x7f553c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:50.111 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.100+0000 7f555e136700 1 --2- 172.21.15.43:0/2910007101 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f553c06ace0 0x7f553c06d190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5554009ab0 tx=0x7f5554008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:50.112 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.104+0000 7f554effd700 1 -- 172.21.15.43:0/2910007101 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f55580b5510 con 0x7f55580d10f0 2023-06-28T20:08:50.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.297+0000 7f556039a700 1 -- 172.21.15.43:0/2910007101 --> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f55580cc3f0 con 0x7f553c06ace0 2023-06-28T20:08:50.303 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.300+0000 7f554effd700 1 -- 172.21.15.43:0/2910007101 <== mgr.4101 v2:172.21.15.43:6832/406144153 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72956 (secure 0 0 0) 0x7f5554011750 con 0x7f553c06ace0 2023-06-28T20:08:50.304 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:50.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.302+0000 7f556039a700 1 -- 172.21.15.43:0/2910007101 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f553c06ace0 msgr2=0x7f553c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:50.305 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.302+0000 7f556039a700 1 --2- 172.21.15.43:0/2910007101 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f553c06ace0 0x7f553c06d190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5554009ab0 tx=0x7f5554008040 comp rx=0 tx=0).stop 2023-06-28T20:08:50.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.302+0000 7f556039a700 1 -- 172.21.15.43:0/2910007101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55580d10f0 msgr2=0x7f55580c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:50.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.302+0000 7f556039a700 1 --2- 172.21.15.43:0/2910007101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55580d10f0 0x7f55580c7db0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f555000b680 tx=0x7f5550004ad0 comp rx=0 tx=0).stop 2023-06-28T20:08:50.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.302+0000 7f556039a700 1 -- 172.21.15.43:0/2910007101 shutdown_connections 2023-06-28T20:08:50.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.302+0000 7f556039a700 1 --2- 172.21.15.43:0/2910007101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55580d10f0 0x7f55580c7db0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:50.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.302+0000 7f556039a700 1 --2- 172.21.15.43:0/2910007101 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f553c06ace0 0x7f553c06d190 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:50.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.302+0000 7f556039a700 1 --2- 172.21.15.43:0/2910007101 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f55580cbab0 0x7f55580ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:50.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.302+0000 7f556039a700 1 --2- 172.21.15.43:0/2910007101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f55580cedf0 0x7f5558158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:50.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.302+0000 7f556039a700 1 -- 172.21.15.43:0/2910007101 >> 172.21.15.43:0/2910007101 conn(0x7f55580664b0 msgr2=0x7f55580ad930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:50.307 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.302+0000 7f556039a700 1 -- 172.21.15.43:0/2910007101 shutdown_connections 2023-06-28T20:08:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.302+0000 7f556039a700 1 -- 172.21.15.43:0/2910007101 wait complete. 2023-06-28T20:08:50.308 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-06-28T20:08:50.317 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2023-06-28T20:08:50.009053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220228,"kb_used_data":4612,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754492,"statfs":{"total":773094113280,"available":772868599808,"internally_reserved":0,"allocated":4722688,"data_stored":2649231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.928536"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.226142+0000","last_change":"2023-06-28T20:07:15.815425+0000","last_active":"2023-06-28T20:08:42.226142+0000","last_peered":"2023-06-28T20:08:42.226142+0000","last_clean":"2023-06-28T20:08:42.226142+0000","last_became_active":"2023-06-28T20:07:15.815163+0000","last_became_peered":"2023-06-28T20:07:15.815163+0000","last_unstale":"2023-06-28T20:08:42.226142+0000","last_undegraded":"2023-06-28T20:08:42.226142+0000","last_fullsized":"2023-06-28T20:08:42.226142+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:54:07.698762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":49,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.225754+0000","last_change":"2023-06-28T20:07:15.303675+0000","last_active":"2023-06-28T20:08:42.225754+0000","last_peered":"2023-06-28T20:08:42.225754+0000","last_clean":"2023-06-28T20:08:42.225754+0000","last_became_active":"2023-06-28T20:07:15.303350+0000","last_became_peered":"2023-06-28T20:07:15.303350+0000","last_unstale":"2023-06-28T20:08:42.225754+0000","last_undegraded":"2023-06-28T20:08:42.225754+0000","last_fullsized":"2023-06-28T20:08:42.225754+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:51:41.786605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.225001+0000","last_change":"2023-06-28T20:07:15.303565+0000","last_active":"2023-06-28T20:08:42.225001+0000","last_peered":"2023-06-28T20:08:42.225001+0000","last_clean":"2023-06-28T20:08:42.225001+0000","last_became_active":"2023-06-28T20:07:15.303200+0000","last_became_peered":"2023-06-28T20:07:15.303200+0000","last_unstale":"2023-06-28T20:08:42.225001+0000","last_undegraded":"2023-06-28T20:08:42.225001+0000","last_fullsized":"2023-06-28T20:08:42.225001+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:02:44.229956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":189,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.094410+0000","last_change":"2023-06-28T20:07:15.304921+0000","last_active":"2023-06-28T20:08:42.094410+0000","last_peered":"2023-06-28T20:08:42.094410+0000","last_clean":"2023-06-28T20:08:42.094410+0000","last_became_active":"2023-06-28T20:07:15.303959+0000","last_became_peered":"2023-06-28T20:07:15.303959+0000","last_unstale":"2023-06-28T20:08:42.094410+0000","last_undegraded":"2023-06-28T20:08:42.094410+0000","last_fullsized":"2023-06-28T20:08:42.094410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:07:09.587950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":41,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.094663+0000","last_change":"2023-06-28T20:07:15.899048+0000","last_active":"2023-06-28T20:08:42.094663+0000","last_peered":"2023-06-28T20:08:42.094663+0000","last_clean":"2023-06-28T20:08:42.094663+0000","last_became_active":"2023-06-28T20:07:15.898766+0000","last_became_peered":"2023-06-28T20:07:15.898766+0000","last_unstale":"2023-06-28T20:08:42.094663+0000","last_undegraded":"2023-06-28T20:08:42.094663+0000","last_fullsized":"2023-06-28T20:08:42.094663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:32:41.079504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":39,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.094199+0000","last_change":"2023-06-28T20:07:15.899229+0000","last_active":"2023-06-28T20:08:42.094199+0000","last_peered":"2023-06-28T20:08:42.094199+0000","last_clean":"2023-06-28T20:08:42.094199+0000","last_became_active":"2023-06-28T20:07:15.898708+0000","last_became_peered":"2023-06-28T20:07:15.898708+0000","last_unstale":"2023-06-28T20:08:42.094199+0000","last_undegraded":"2023-06-28T20:08:42.094199+0000","last_fullsized":"2023-06-28T20:08:42.094199+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:11:32.901441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.227358+0000","last_change":"2023-06-28T20:07:15.898966+0000","last_active":"2023-06-28T20:08:42.227358+0000","last_peered":"2023-06-28T20:08:42.227358+0000","last_clean":"2023-06-28T20:08:42.227358+0000","last_became_active":"2023-06-28T20:07:15.898704+0000","last_became_peered":"2023-06-28T20:07:15.898704+0000","last_unstale":"2023-06-28T20:08:42.227358+0000","last_undegraded":"2023-06-28T20:08:42.227358+0000","last_fullsized":"2023-06-28T20:08:42.227358+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:19:46.078630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":53,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.095830+0000","last_change":"2023-06-28T20:07:15.899207+0000","last_active":"2023-06-28T20:08:42.095830+0000","last_peered":"2023-06-28T20:08:42.095830+0000","last_clean":"2023-06-28T20:08:42.095830+0000","last_became_active":"2023-06-28T20:07:15.898908+0000","last_became_peered":"2023-06-28T20:07:15.898908+0000","last_unstale":"2023-06-28T20:08:42.095830+0000","last_undegraded":"2023-06-28T20:08:42.095830+0000","last_fullsized":"2023-06-28T20:08:42.095830+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:58:47.410109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":53,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.095885+0000","last_change":"2023-06-28T20:07:15.303873+0000","last_active":"2023-06-28T20:08:42.095885+0000","last_peered":"2023-06-28T20:08:42.095885+0000","last_clean":"2023-06-28T20:08:42.095885+0000","last_became_active":"2023-06-28T20:07:15.303457+0000","last_became_peered":"2023-06-28T20:07:15.303457+0000","last_unstale":"2023-06-28T20:08:42.095885+0000","last_undegraded":"2023-06-28T20:08:42.095885+0000","last_fullsized":"2023-06-28T20:08:42.095885+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:20:41.981562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":1036288,"data_stored":774050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61899999999999999}]},{"osd":1,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.42599999999999999}]},{"osd":2,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.57399999999999995}]},{"osd":3,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.64900000000000002}]},{"osd":4,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.47199999999999998}]},{"osd":5,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.54300000000000004}]},{"osd":6,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.502},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.66300000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":183682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":9.0069999999999997,"5min":9.0069999999999997,"15min":9.0069999999999997},"last":0.57099999999999995}]},{"osd":1,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":3.036,"5min":3.036,"15min":3.036},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.64800000000000002}]},{"osd":2,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.66700000000000004}]},{"osd":3,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":9.0459999999999994,"5min":9.0459999999999994,"15min":9.0459999999999994},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.59499999999999997}]},{"osd":4,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.38900000000000001}]},{"osd":5,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.372},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.41999999999999998}]},{"osd":7,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.44800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":1036288,"data_stored":774050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.49399999999999999}]},{"osd":2,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.47099999999999997}]},{"osd":3,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.56499999999999995}]},{"osd":4,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.80000000000000004}]},{"osd":5,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.745}]},{"osd":6,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.82199999999999995}]},{"osd":7,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.78300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":438272,"data_stored":182721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.53400000000000003}]},{"osd":2,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.442,"5min":0.442,"15min":0.442},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.58699999999999997}]},{"osd":3,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.66700000000000004}]},{"osd":4,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":1}]},{"osd":5,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.91300000000000003}]},{"osd":6,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.95699999999999996}]},{"osd":7,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.93700000000000006}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":183682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":16.044,"5min":16.044,"15min":16.044},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.51200000000000001}]},{"osd":1,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":11.028,"5min":11.028,"15min":11.028},"last":0.38800000000000001},{"interface":"front","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":16.077000000000002,"5min":16.077000000000002,"15min":16.077000000000002},"last":0.35299999999999998}]},{"osd":3,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.47399999999999998},{"interface":"front","average":{"1min":1.319,"5min":1.319,"15min":1.319},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":16.004000000000001,"5min":16.004000000000001,"15min":16.004000000000001},"last":0.439}]},{"osd":4,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":9.327,"5min":9.327,"15min":9.327},"last":0.66600000000000004},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":10.964,"5min":10.964,"15min":10.964},"last":0.71599999999999997}]},{"osd":5,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":10.923,"5min":10.923,"15min":10.923},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.77200000000000002}]},{"osd":6,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":9.282,"5min":9.282,"15min":9.282},"last":0.63600000000000001}]},{"osd":7,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":10.996,"5min":10.996,"15min":10.996},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.73599999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":183682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.496},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":5.0250000000000004,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":0.54800000000000004}]},{"osd":1,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":5.2309999999999999,"5min":5.2309999999999999,"15min":5.2309999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":5.3250000000000002,"5min":5.3250000000000002,"15min":5.3250000000000002},"last":0.63700000000000001}]},{"osd":2,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":15.927,"5min":15.927,"15min":15.927},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.66400000000000003}]},{"osd":4,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.79100000000000004}]},{"osd":5,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":4.9939999999999998,"5min":4.9939999999999998,"15min":4.9939999999999998},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.68999999999999995}]},{"osd":6,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":4.9619999999999997,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.72299999999999998}]},{"osd":7,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":4.9210000000000003,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.77400000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":183682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":9.1809999999999992,"5min":9.1809999999999992,"15min":9.1809999999999992},"last":0.68899999999999995}]},{"osd":1,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67700000000000005}]},{"osd":2,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":9.0700000000000003,"5min":9.0700000000000003,"15min":9.0700000000000003},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.80000000000000004}]},{"osd":3,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.78400000000000003}]},{"osd":5,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.64400000000000002}]},{"osd":6,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.76000000000000001}]},{"osd":7,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.745}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":183682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.51300000000000001}]},{"osd":1,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.46500000000000002}]},{"osd":2,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.629}]},{"osd":3,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":5.6139999999999999,"5min":5.6139999999999999,"15min":5.6139999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":5.6500000000000004,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.58099999999999996}]},{"osd":4,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.59199999999999997}]},{"osd":6,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.68300000000000005}]},{"osd":7,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.61299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-28T20:08:50.321 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-28T20:08:50.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.443+0000 7f715a393700 1 Processor -- start 2023-06-28T20:08:50.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.443+0000 7f715a393700 1 -- start start 2023-06-28T20:08:50.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.444+0000 7f715a393700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f71540718e0 0x7f71540ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:50.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.445+0000 7f715a393700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71540cd2c0 0x7f71540cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:50.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.445+0000 7f715a393700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71540d0430 con 0x7f7154159480 2023-06-28T20:08:50.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.445+0000 7f715a393700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71540d0710 con 0x7f71540718e0 2023-06-28T20:08:50.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.445+0000 7f715a393700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71540d0250 con 0x7f71540cd2c0 2023-06-28T20:08:50.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.445+0000 7f71537fe700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f71540718e0 0x7f71540ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:50.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.445+0000 7f7158930700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71540cd2c0 0x7f71540cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:50.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.445+0000 7f71537fe700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f71540718e0 0x7f71540ccd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.43:41734/0 (socket says 172.21.15.43:41734) 2023-06-28T20:08:50.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.445+0000 7f7153fff700 1 --1- >> v1:172.21.15.178:6789/0 conn(0x7f7154159480 0x7f71540cfb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:43978/0 (socket says 172.21.15.43:43978) 2023-06-28T20:08:50.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.445+0000 7f71537fe700 1 -- 172.21.15.43:0/1052129462 learned_addr learned my addr 172.21.15.43:0/1052129462 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:50.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.445+0000 7f7158930700 1 -- 172.21.15.43:0/1052129462 >> v2:172.21.15.178:3300/0 conn(0x7f71540718e0 msgr2=0x7f71540ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:50.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.445+0000 7f7158930700 1 --2- 172.21.15.43:0/1052129462 >> v2:172.21.15.178:3300/0 conn(0x7f71540718e0 0x7f71540ccd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:50.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.445+0000 7f7158930700 1 -- 172.21.15.43:0/1052129462 >> v1:172.21.15.178:6789/0 conn(0x7f7154159480 legacy=0x7f71540cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:50.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.445+0000 7f7158930700 1 -- 172.21.15.43:0/1052129462 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71540cc2a0 con 0x7f71540cd2c0 2023-06-28T20:08:50.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.445+0000 7f7152ffd700 1 -- 172.21.15.43:0/1052129462 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1429680945 0 0) 0x7f71540d0430 con 0x7f7154159480 2023-06-28T20:08:50.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.445+0000 7f71537fe700 1 --2- 172.21.15.43:0/1052129462 >> v2:172.21.15.178:3300/0 conn(0x7f71540718e0 0x7f71540ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:08:50.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.446+0000 7f7158930700 1 --2- 172.21.15.43:0/1052129462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71540cd2c0 0x7f71540cdb50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f714c009d90 tx=0x7f714c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34760c7912823f16 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:50.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.446+0000 7f7152ffd700 1 -- 172.21.15.43:0/1052129462 <== mon.2 v2:172.21.15.178:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c00e070 con 0x7f71540cd2c0 2023-06-28T20:08:50.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.446+0000 7f7152ffd700 1 -- 172.21.15.43:0/1052129462 <== mon.2 v2:172.21.15.178:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714c00bb20 con 0x7f71540cd2c0 2023-06-28T20:08:50.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.446+0000 7f7152ffd700 1 -- 172.21.15.43:0/1052129462 <== mon.2 v2:172.21.15.178:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c012430 con 0x7f71540cd2c0 2023-06-28T20:08:50.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.446+0000 7f715a393700 1 -- 172.21.15.43:0/1052129462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71540cd2c0 msgr2=0x7f71540cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:50.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.446+0000 7f715a393700 1 --2- 172.21.15.43:0/1052129462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71540cd2c0 0x7f71540cdb50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f714c009d90 tx=0x7f714c00b330 comp rx=0 tx=0).stop 2023-06-28T20:08:50.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.446+0000 7f715a393700 1 -- 172.21.15.43:0/1052129462 shutdown_connections 2023-06-28T20:08:50.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.446+0000 7f715a393700 1 --2- 172.21.15.43:0/1052129462 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f71540cd2c0 0x7f71540cdb50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:50.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.446+0000 7f715a393700 1 --2- 172.21.15.43:0/1052129462 >> v2:172.21.15.178:3300/0 conn(0x7f71540718e0 0x7f71540ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:50.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.446+0000 7f715a393700 1 -- 172.21.15.43:0/1052129462 >> 172.21.15.43:0/1052129462 conn(0x7f715406b000 msgr2=0x7f715406b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:50.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.446+0000 7f715a393700 1 -- 172.21.15.43:0/1052129462 shutdown_connections 2023-06-28T20:08:50.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.446+0000 7f715a393700 1 -- 172.21.15.43:0/1052129462 wait complete. 2023-06-28T20:08:50.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.448+0000 7f715a393700 1 Processor -- start 2023-06-28T20:08:50.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.448+0000 7f715a393700 1 -- start start 2023-06-28T20:08:50.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.449+0000 7f715a393700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71540718e0 0x7f7154156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:50.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.449+0000 7f715a393700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71540cd2c0 0x7f71541571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:50.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.449+0000 7f715a393700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7154159480 0x7f71540b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:50.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.449+0000 7f715a393700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71540c70e0 con 0x7f71540cd2c0 2023-06-28T20:08:50.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.449+0000 7f715a393700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71540c73c0 con 0x7f71540718e0 2023-06-28T20:08:50.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.449+0000 7f715a393700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71540c76a0 con 0x7f7154159480 2023-06-28T20:08:50.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.449+0000 7f71537fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71540cd2c0 0x7f71541571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:50.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.449+0000 7f71537fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71540cd2c0 0x7f71541571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55676/0 (socket says 172.21.15.43:55676) 2023-06-28T20:08:50.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.449+0000 7f71537fe700 1 -- 172.21.15.43:0/1163978019 learned_addr learned my addr 172.21.15.43:0/1163978019 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:50.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.449+0000 7f7153fff700 1 --2- 172.21.15.43:0/1163978019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71540718e0 0x7f7154156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:50.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.449+0000 7f7158930700 1 --2- 172.21.15.43:0/1163978019 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7154159480 0x7f71540b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:50.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.450+0000 7f71537fe700 1 -- 172.21.15.43:0/1163978019 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7154159480 msgr2=0x7f71540b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:50.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.450+0000 7f71537fe700 1 --2- 172.21.15.43:0/1163978019 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7154159480 0x7f71540b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:50.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.450+0000 7f71537fe700 1 -- 172.21.15.43:0/1163978019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71540718e0 msgr2=0x7f7154156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:50.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.450+0000 7f71537fe700 1 --2- 172.21.15.43:0/1163978019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71540718e0 0x7f7154156c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:50.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.450+0000 7f71537fe700 1 -- 172.21.15.43:0/1163978019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713c009550 con 0x7f71540cd2c0 2023-06-28T20:08:50.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.450+0000 7f71537fe700 1 --2- 172.21.15.43:0/1163978019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71540cd2c0 0x7f71541571d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f713c010bc0 tx=0x7f713c010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:50.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.450+0000 7f7150ff9700 1 -- 172.21.15.43:0/1163978019 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c00ed20 con 0x7f71540cd2c0 2023-06-28T20:08:50.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.450+0000 7f7150ff9700 1 -- 172.21.15.43:0/1163978019 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713c00ee80 con 0x7f71540cd2c0 2023-06-28T20:08:50.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.450+0000 7f715a393700 1 -- 172.21.15.43:0/1163978019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f714c009a70 con 0x7f71540cd2c0 2023-06-28T20:08:50.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.450+0000 7f7150ff9700 1 -- 172.21.15.43:0/1163978019 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c011650 con 0x7f71540cd2c0 2023-06-28T20:08:50.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.450+0000 7f715a393700 1 -- 172.21.15.43:0/1163978019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71540c7c60 con 0x7f71540cd2c0 2023-06-28T20:08:50.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.451+0000 7f715a393700 1 -- 172.21.15.43:0/1163978019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71540c7e20 con 0x7f71540cd2c0 2023-06-28T20:08:50.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.451+0000 7f7150ff9700 1 -- 172.21.15.43:0/1163978019 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f713c0117b0 con 0x7f71540cd2c0 2023-06-28T20:08:50.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.452+0000 7f7150ff9700 1 --2- 172.21.15.43:0/1163978019 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f713406ace0 0x7f713406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:50.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.452+0000 7f7150ff9700 1 -- 172.21.15.43:0/1163978019 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f713c08b430 con 0x7f71540cd2c0 2023-06-28T20:08:50.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.452+0000 7f7153fff700 1 --2- 172.21.15.43:0/1163978019 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f713406ace0 0x7f713406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:50.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.452+0000 7f7153fff700 1 --2- 172.21.15.43:0/1163978019 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f713406ace0 0x7f713406d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7144007fd0 tx=0x7f7144007de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:50.466 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.456+0000 7f7150ff9700 1 -- 172.21.15.43:0/1163978019 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f713c0567e0 con 0x7f71540cd2c0 2023-06-28T20:08:50.658 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.655+0000 7f715a393700 1 -- 172.21.15.43:0/1163978019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71540728b0 con 0x7f71540cd2c0 2023-06-28T20:08:50.659 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.656+0000 7f7150ff9700 1 -- 172.21.15.43:0/1163978019 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+14710 (secure 0 0 0) 0x7f713c059e00 con 0x7f71540cd2c0 2023-06-28T20:08:50.659 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:50.660 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":38,"fsid":"b8fd740d-00b9-4308-92a8-48f5fffe8078","created":"2023-06-28T20:07:02.167102+0000","modified":"2023-06-28T20:08:42.085473+0000","last_up_change":"2023-06-28T20:07:12.279568+0000","last_in_change":"2023-06-28T20:07:04.729846+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-06-28T20:07:13.580969+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-06-28T20:07:14.038358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bd41dbcc-2d80-479d-b0ed-9318bc419195","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6801","nonce":2709258135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6803","nonce":2709258135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6807","nonce":2709258135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2709258135},{"type":"v1","addr":"172.21.15.43:6805","nonce":2709258135}]},"public_addr":"172.21.15.43:6801/2709258135","cluster_addr":"172.21.15.43:6803/2709258135","heartbeat_back_addr":"172.21.15.43:6807/2709258135","heartbeat_front_addr":"172.21.15.43:6805/2709258135","state":["exists","up"]},{"osd":1,"uuid":"82402670-b52b-443d-8d53-a63d02145525","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6809","nonce":894763335}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6811","nonce":894763335}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6815","nonce":894763335}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":894763335},{"type":"v1","addr":"172.21.15.43:6813","nonce":894763335}]},"public_addr":"172.21.15.43:6809/894763335","cluster_addr":"172.21.15.43:6811/894763335","heartbeat_back_addr":"172.21.15.43:6815/894763335","heartbeat_front_addr":"172.21.15.43:6813/894763335","state":["exists","up"]},{"osd":2,"uuid":"27713d73-4e63-4676-af2f-8e6753b8967e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6825","nonce":1523179953}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6827","nonce":1523179953}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6831","nonce":1523179953}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1523179953},{"type":"v1","addr":"172.21.15.43:6829","nonce":1523179953}]},"public_addr":"172.21.15.43:6825/1523179953","cluster_addr":"172.21.15.43:6827/1523179953","heartbeat_back_addr":"172.21.15.43:6831/1523179953","heartbeat_front_addr":"172.21.15.43:6829/1523179953","state":["exists","up"]},{"osd":3,"uuid":"bcf507c8-d64e-41ee-97ef-102353dbbeeb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6817","nonce":2901255345}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6819","nonce":2901255345}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6823","nonce":2901255345}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2901255345},{"type":"v1","addr":"172.21.15.43:6821","nonce":2901255345}]},"public_addr":"172.21.15.43:6817/2901255345","cluster_addr":"172.21.15.43:6819/2901255345","heartbeat_back_addr":"172.21.15.43:6823/2901255345","heartbeat_front_addr":"172.21.15.43:6821/2901255345","state":["exists","up"]},{"osd":4,"uuid":"97b48458-4459-43be-bc4a-8e5100067cd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6822","nonce":2918661202}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6826","nonce":2918661202}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6831","nonce":2918661202}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":2918661202},{"type":"v1","addr":"172.21.15.178:6829","nonce":2918661202}]},"public_addr":"172.21.15.178:6822/2918661202","cluster_addr":"172.21.15.178:6826/2918661202","heartbeat_back_addr":"172.21.15.178:6831/2918661202","heartbeat_front_addr":"172.21.15.178:6829/2918661202","state":["exists","up"]},{"osd":5,"uuid":"b9d323dd-4ed4-4710-8eb6-713385fafe44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6809","nonce":3916491871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6811","nonce":3916491871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6815","nonce":3916491871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":3916491871},{"type":"v1","addr":"172.21.15.178:6813","nonce":3916491871}]},"public_addr":"172.21.15.178:6809/3916491871","cluster_addr":"172.21.15.178:6811/3916491871","heartbeat_back_addr":"172.21.15.178:6815/3916491871","heartbeat_front_addr":"172.21.15.178:6813/3916491871","state":["exists","up"]},{"osd":6,"uuid":"e0f5c961-804d-43eb-9668-00b880ddc51b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6801","nonce":2113985646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6803","nonce":2113985646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6807","nonce":2113985646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2113985646},{"type":"v1","addr":"172.21.15.178:6805","nonce":2113985646}]},"public_addr":"172.21.15.178:6801/2113985646","cluster_addr":"172.21.15.178:6803/2113985646","heartbeat_back_addr":"172.21.15.178:6807/2113985646","heartbeat_front_addr":"172.21.15.178:6805/2113985646","state":["exists","up"]},{"osd":7,"uuid":"d6e609c6-b6bc-489b-b844-92552a86b88e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6817","nonce":3953458472}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6819","nonce":3953458472}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6825","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6827","nonce":3953458472}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6821","nonce":3953458472},{"type":"v1","addr":"172.21.15.178:6823","nonce":3953458472}]},"public_addr":"172.21.15.178:6817/3953458472","cluster_addr":"172.21.15.178:6819/3953458472","heartbeat_back_addr":"172.21.15.178:6827/3953458472","heartbeat_front_addr":"172.21.15.178:6823/3953458472","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.927629+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.854580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.883548+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.877813+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:10.027486+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.979702+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.922648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-28T20:07:09.969348+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6837/1096784002":"2023-06-29T20:08:38.905088+0000","172.21.15.43:6835/1096784002":"2023-06-29T20:08:38.905088+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-28T20:08:50.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.658+0000 7f715a393700 1 -- 172.21.15.43:0/1163978019 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f713406ace0 msgr2=0x7f713406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:50.661 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.658+0000 7f715a393700 1 --2- 172.21.15.43:0/1163978019 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f713406ace0 0x7f713406d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7144007fd0 tx=0x7f7144007de0 comp rx=0 tx=0).stop 2023-06-28T20:08:50.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.658+0000 7f715a393700 1 -- 172.21.15.43:0/1163978019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71540cd2c0 msgr2=0x7f71541571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:50.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.658+0000 7f715a393700 1 --2- 172.21.15.43:0/1163978019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71540cd2c0 0x7f71541571d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f713c010bc0 tx=0x7f713c010ed0 comp rx=0 tx=0).stop 2023-06-28T20:08:50.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.658+0000 7f715a393700 1 -- 172.21.15.43:0/1163978019 shutdown_connections 2023-06-28T20:08:50.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.658+0000 7f715a393700 1 --2- 172.21.15.43:0/1163978019 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f713406ace0 0x7f713406d190 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:50.662 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.658+0000 7f715a393700 1 --2- 172.21.15.43:0/1163978019 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7154159480 0x7f71540b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:50.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.658+0000 7f715a393700 1 --2- 172.21.15.43:0/1163978019 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71540718e0 0x7f7154156c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:50.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.658+0000 7f715a393700 1 --2- 172.21.15.43:0/1163978019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71540cd2c0 0x7f71541571d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:50.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.658+0000 7f715a393700 1 -- 172.21.15.43:0/1163978019 >> 172.21.15.43:0/1163978019 conn(0x7f715406b000 msgr2=0x7f71540b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:50.663 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.658+0000 7f715a393700 1 -- 172.21.15.43:0/1163978019 shutdown_connections 2023-06-28T20:08:50.664 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:50.658+0000 7f715a393700 1 -- 172.21.15.43:0/1163978019 wait complete. 2023-06-28T20:08:51.675 INFO:tasks.ceph:Scrubbing osd.0 2023-06-28T20:08:51.676 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-06-28T20:08:51.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.817+0000 7fae34e9b700 1 Processor -- start 2023-06-28T20:08:51.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.817+0000 7fae34e9b700 1 -- start start 2023-06-28T20:08:51.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.818+0000 7fae34e9b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae3006d580 0x7fae3006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:51.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.818+0000 7fae34e9b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fae300cfba0 0x7fae3006b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:51.823 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.818+0000 7fae34e9b700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3006b970 con 0x7fae3006d950 2023-06-28T20:08:51.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.818+0000 7fae34e9b700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae3006bc50 con 0x7fae300cfba0 2023-06-28T20:08:51.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.818+0000 7fae34e9b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3006bf30 con 0x7fae3006d580 2023-06-28T20:08:51.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.819+0000 7fae2dd9b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fae300cfba0 0x7fae3006b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:51.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.819+0000 7fae2ed9d700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fae3006d950 0x7fae3015c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44872/0 (socket says 172.21.15.43:44872) 2023-06-28T20:08:51.824 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.819+0000 7fae2ed9d700 1 -- 172.21.15.43:0/4018789345 learned_addr learned my addr 172.21.15.43:0/4018789345 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:51.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.819+0000 7fae2e59c700 1 --2- 172.21.15.43:0/4018789345 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae3006d580 0x7fae3006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:51.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.819+0000 7fae2dd9b700 1 -- 172.21.15.43:0/4018789345 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae3006d580 msgr2=0x7fae3006ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:51.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.819+0000 7fae2dd9b700 1 --2- 172.21.15.43:0/4018789345 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae3006d580 0x7fae3006ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:51.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.819+0000 7fae2dd9b700 1 -- 172.21.15.43:0/4018789345 >> v1:172.21.15.43:6789/0 conn(0x7fae3006d950 legacy=0x7fae3015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:51.825 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.819+0000 7fae2dd9b700 1 -- 172.21.15.43:0/4018789345 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3006c210 con 0x7fae300cfba0 2023-06-28T20:08:51.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.819+0000 7fae2d59a700 1 -- 172.21.15.43:0/4018789345 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3392559810 0 0) 0x7fae3006b970 con 0x7fae3006d950 2023-06-28T20:08:51.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.819+0000 7fae2dd9b700 1 --2- 172.21.15.43:0/4018789345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae300cfba0 0x7fae3006b000 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fae18009840 tx=0x7fae18009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4365d79318c1d539 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:51.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.819+0000 7fae2d59a700 1 -- 172.21.15.43:0/4018789345 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae18011830 con 0x7fae300cfba0 2023-06-28T20:08:51.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.819+0000 7fae2d59a700 1 -- 172.21.15.43:0/4018789345 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae18011990 con 0x7fae300cfba0 2023-06-28T20:08:51.826 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.819+0000 7fae2d59a700 1 -- 172.21.15.43:0/4018789345 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae18011d20 con 0x7fae300cfba0 2023-06-28T20:08:51.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.820+0000 7fae34e9b700 1 -- 172.21.15.43:0/4018789345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae300cfba0 msgr2=0x7fae3006b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:51.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.820+0000 7fae34e9b700 1 --2- 172.21.15.43:0/4018789345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae300cfba0 0x7fae3006b000 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fae18009840 tx=0x7fae18009b50 comp rx=0 tx=0).stop 2023-06-28T20:08:51.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.820+0000 7fae34e9b700 1 -- 172.21.15.43:0/4018789345 shutdown_connections 2023-06-28T20:08:51.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.820+0000 7fae34e9b700 1 --2- 172.21.15.43:0/4018789345 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae3006d580 0x7fae3006ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:51.827 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.820+0000 7fae34e9b700 1 --2- 172.21.15.43:0/4018789345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae300cfba0 0x7fae3006b000 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:51.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.820+0000 7fae34e9b700 1 -- 172.21.15.43:0/4018789345 >> 172.21.15.43:0/4018789345 conn(0x7fae30099f30 msgr2=0x7fae300d0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:51.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.820+0000 7fae34e9b700 1 -- 172.21.15.43:0/4018789345 shutdown_connections 2023-06-28T20:08:51.828 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.820+0000 7fae34e9b700 1 -- 172.21.15.43:0/4018789345 wait complete. 2023-06-28T20:08:51.830 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.821+0000 7fae34e9b700 1 Processor -- start 2023-06-28T20:08:51.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.821+0000 7fae34e9b700 1 -- start start 2023-06-28T20:08:51.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.822+0000 7fae34e9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3006d580 0x7fae30159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:51.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.822+0000 7fae34e9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae3006d950 0x7fae3015c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:51.831 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.822+0000 7fae34e9b700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae300cfba0 0x7fae3015c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:51.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.822+0000 7fae34e9b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3009bac0 con 0x7fae3006d580 2023-06-28T20:08:51.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.822+0000 7fae34e9b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3009ae40 con 0x7fae3006d950 2023-06-28T20:08:51.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.822+0000 7fae34e9b700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3009b120 con 0x7fae300cfba0 2023-06-28T20:08:51.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.822+0000 7fae2e59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3006d580 0x7fae30159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:51.832 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.822+0000 7fae2e59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3006d580 0x7fae30159450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55688/0 (socket says 172.21.15.43:55688) 2023-06-28T20:08:51.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.822+0000 7fae2e59c700 1 -- 172.21.15.43:0/465298352 learned_addr learned my addr 172.21.15.43:0/465298352 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:51.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.823+0000 7fae2dd9b700 1 --2- 172.21.15.43:0/465298352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae3006d950 0x7fae3015c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:51.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.823+0000 7fae2ed9d700 1 --2- 172.21.15.43:0/465298352 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae300cfba0 0x7fae3015c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:51.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.823+0000 7fae2e59c700 1 -- 172.21.15.43:0/465298352 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae300cfba0 msgr2=0x7fae3015c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:51.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.823+0000 7fae2e59c700 1 --2- 172.21.15.43:0/465298352 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fae300cfba0 0x7fae3015c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:51.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.823+0000 7fae2e59c700 1 -- 172.21.15.43:0/465298352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae3006d950 msgr2=0x7fae3015c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:51.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.823+0000 7fae2e59c700 1 --2- 172.21.15.43:0/465298352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae3006d950 0x7fae3015c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:51.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.823+0000 7fae2e59c700 1 -- 172.21.15.43:0/465298352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae180094a0 con 0x7fae3006d580 2023-06-28T20:08:51.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.823+0000 7fae2e59c700 1 --2- 172.21.15.43:0/465298352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3006d580 0x7fae30159450 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fae2800efc0 tx=0x7fae2800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:51.834 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.823+0000 7fae26ffd700 1 -- 172.21.15.43:0/465298352 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2800cdc0 con 0x7fae3006d580 2023-06-28T20:08:51.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.823+0000 7fae26ffd700 1 -- 172.21.15.43:0/465298352 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2800f040 con 0x7fae3006d580 2023-06-28T20:08:51.835 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.823+0000 7fae34e9b700 1 -- 172.21.15.43:0/465298352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3009b400 con 0x7fae3006d580 2023-06-28T20:08:51.836 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.823+0000 7fae26ffd700 1 -- 172.21.15.43:0/465298352 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae280146a0 con 0x7fae3006d580 2023-06-28T20:08:51.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.823+0000 7fae34e9b700 1 -- 172.21.15.43:0/465298352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae30159990 con 0x7fae3006d580 2023-06-28T20:08:51.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.824+0000 7fae34e9b700 1 -- 172.21.15.43:0/465298352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fae3006cf50 con 0x7fae3006d580 2023-06-28T20:08:51.837 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.825+0000 7fae26ffd700 1 -- 172.21.15.43:0/465298352 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fae28014800 con 0x7fae3006d580 2023-06-28T20:08:51.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.825+0000 7fae26ffd700 1 --2- 172.21.15.43:0/465298352 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fae1006ace0 0x7fae1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:51.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.825+0000 7fae26ffd700 1 -- 172.21.15.43:0/465298352 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fae2808a690 con 0x7fae3006d580 2023-06-28T20:08:51.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.825+0000 7fae2dd9b700 1 --2- 172.21.15.43:0/465298352 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fae1006ace0 0x7fae1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:51.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.825+0000 7fae26ffd700 1 --2- 172.21.15.43:0/465298352 >> [v2:172.21.15.43:6800/2709258135,v1:172.21.15.43:6801/2709258135] conn(0x7fae10071150 0x7fae10073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:51.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.825+0000 7fae26ffd700 1 -- 172.21.15.43:0/465298352 --> [v2:172.21.15.43:6800/2709258135,v1:172.21.15.43:6801/2709258135] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae10073c30 con 0x7fae10071150 2023-06-28T20:08:51.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.825+0000 7fae26ffd700 1 -- 172.21.15.43:0/465298352 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae2808aac0 con 0x7fae3006d580 2023-06-28T20:08:51.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.825+0000 7fae2ed9d700 1 --2- 172.21.15.43:0/465298352 >> [v2:172.21.15.43:6800/2709258135,v1:172.21.15.43:6801/2709258135] conn(0x7fae10071150 0x7fae10073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:51.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.825+0000 7fae2dd9b700 1 --2- 172.21.15.43:0/465298352 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fae1006ace0 0x7fae1006d190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fae300cff70 tx=0x7fae1801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:51.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.826+0000 7fae2ed9d700 1 --2- 172.21.15.43:0/465298352 >> [v2:172.21.15.43:6800/2709258135,v1:172.21.15.43:6801/2709258135] conn(0x7fae10071150 0x7fae10073560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:51.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.826+0000 7fae26ffd700 1 -- 172.21.15.43:0/465298352 <== osd.0 v2:172.21.15.43:6800/2709258135 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fae10073c30 con 0x7fae10071150 2023-06-28T20:08:51.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.841+0000 7fae34e9b700 1 -- 172.21.15.43:0/465298352 --> [v2:172.21.15.43:6800/2709258135,v1:172.21.15.43:6801/2709258135] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fae3006c4f0 con 0x7fae10071150 2023-06-28T20:08:51.844 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.841+0000 7fae26ffd700 1 -- 172.21.15.43:0/465298352 <== osd.0 v2:172.21.15.43:6800/2709258135 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fae20005a20 con 0x7fae10071150 2023-06-28T20:08:51.844 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-06-28T20:08:51.845 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-28T20:08:51.845 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-06-28T20:08:51.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.841+0000 7fae34e9b700 1 -- 172.21.15.43:0/465298352 >> [v2:172.21.15.43:6800/2709258135,v1:172.21.15.43:6801/2709258135] conn(0x7fae10071150 msgr2=0x7fae10073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:51.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.841+0000 7fae34e9b700 1 --2- 172.21.15.43:0/465298352 >> [v2:172.21.15.43:6800/2709258135,v1:172.21.15.43:6801/2709258135] conn(0x7fae10071150 0x7fae10073560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:51.846 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.841+0000 7fae34e9b700 1 -- 172.21.15.43:0/465298352 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fae1006ace0 msgr2=0x7fae1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.841+0000 7fae34e9b700 1 --2- 172.21.15.43:0/465298352 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fae1006ace0 0x7fae1006d190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fae300cff70 tx=0x7fae1801c040 comp rx=0 tx=0).stop 2023-06-28T20:08:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.841+0000 7fae34e9b700 1 -- 172.21.15.43:0/465298352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3006d580 msgr2=0x7fae30159450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.841+0000 7fae34e9b700 1 --2- 172.21.15.43:0/465298352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3006d580 0x7fae30159450 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fae2800efc0 tx=0x7fae2800c3a0 comp rx=0 tx=0).stop 2023-06-28T20:08:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.841+0000 7fae2ed9d700 1 -- 172.21.15.43:0/465298352 reap_dead start 2023-06-28T20:08:51.847 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.841+0000 7fae34e9b700 1 -- 172.21.15.43:0/465298352 shutdown_connections 2023-06-28T20:08:51.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.841+0000 7fae34e9b700 1 -- 172.21.15.43:0/465298352 >> 172.21.15.43:0/465298352 conn(0x7fae30099f30 msgr2=0x7fae300cffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:51.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.841+0000 7fae34e9b700 1 -- 172.21.15.43:0/465298352 shutdown_connections 2023-06-28T20:08:51.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.841+0000 7fae34e9b700 1 -- 172.21.15.43:0/465298352 wait complete. 2023-06-28T20:08:51.857 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-06-28T20:08:51.983 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.980+0000 7fcdf2a43700 1 Processor -- start 2023-06-28T20:08:51.984 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.980+0000 7fcdf2a43700 1 -- start start 2023-06-28T20:08:51.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.981+0000 7fcdf2a43700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fcdec0b20e0 0x7fcdec15d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:51.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.981+0000 7fcdf2a43700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec1596a0 0x7fcdec15d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:51.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.981+0000 7fcdf2a43700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdec070b40 con 0x7fcdec1592d0 2023-06-28T20:08:51.988 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.981+0000 7fcdf2a43700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdec070e20 con 0x7fcdec0b20e0 2023-06-28T20:08:51.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.981+0000 7fcdf2a43700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec071100 con 0x7fcdec1596a0 2023-06-28T20:08:51.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.981+0000 7fcdebfff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcdec1592d0 0x7fcdec0b19d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44884/0 (socket says 172.21.15.43:44884) 2023-06-28T20:08:51.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.981+0000 7fcdebfff700 1 -- 172.21.15.43:0/237644505 learned_addr learned my addr 172.21.15.43:0/237644505 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:51.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdeaffd700 1 -- 172.21.15.43:0/237644505 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1639435260 0 0) 0x7fcdec070b40 con 0x7fcdec1592d0 2023-06-28T20:08:51.989 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdeaffd700 1 -- 172.21.15.43:0/237644505 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdd80035b0 con 0x7fcdec1592d0 2023-06-28T20:08:51.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdf0fe0700 1 --2- 172.21.15.43:0/237644505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec1596a0 0x7fcdec15d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:51.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdeb7fe700 1 --2- 172.21.15.43:0/237644505 >> v2:172.21.15.178:3300/0 conn(0x7fcdec0b20e0 0x7fcdec15d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:51.990 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdeaffd700 1 -- 172.21.15.43:0/237644505 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3882505912 0 0) 0x7fcdd80035b0 con 0x7fcdec1592d0 2023-06-28T20:08:51.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdeaffd700 1 -- 172.21.15.43:0/237644505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec1596a0 msgr2=0x7fcdec15d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:51.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdeaffd700 1 --2- 172.21.15.43:0/237644505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec1596a0 0x7fcdec15d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:51.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdeaffd700 1 -- 172.21.15.43:0/237644505 >> v2:172.21.15.178:3300/0 conn(0x7fcdec0b20e0 msgr2=0x7fcdec15d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:51.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdeaffd700 1 --2- 172.21.15.43:0/237644505 >> v2:172.21.15.178:3300/0 conn(0x7fcdec0b20e0 0x7fcdec15d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:51.991 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdeaffd700 1 -- 172.21.15.43:0/237644505 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec0713e0 con 0x7fcdec1592d0 2023-06-28T20:08:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdeaffd700 1 -- 172.21.15.43:0/237644505 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7fcddc001670 con 0x7fcdec1592d0 2023-06-28T20:08:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdeaffd700 1 -- 172.21.15.43:0/237644505 >> v1:172.21.15.43:6789/0 conn(0x7fcdec1592d0 legacy=0x7fcdec0b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdeaffd700 1 --2- 172.21.15.43:0/237644505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec1596a0 0x7fcdd8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdeaffd700 1 --2- 172.21.15.43:0/237644505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdd8003f40 0x7fcdd8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdeaffd700 1 --2- 172.21.15.43:0/237644505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdd8004d90 0x7fcdd8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdeaffd700 1 -- 172.21.15.43:0/237644505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec070e20 con 0x7fcdd8003f40 2023-06-28T20:08:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdeaffd700 1 -- 172.21.15.43:0/237644505 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec071100 con 0x7fcdd8004d90 2023-06-28T20:08:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdeaffd700 1 -- 172.21.15.43:0/237644505 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd8010020 con 0x7fcdec1596a0 2023-06-28T20:08:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdeb7fe700 1 --2- 172.21.15.43:0/237644505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdd8003f40 0x7fcdd8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:51.993 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdf0fe0700 1 --2- 172.21.15.43:0/237644505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdd8004d90 0x7fcdd8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:51.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdebfff700 1 --2- 172.21.15.43:0/237644505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec1596a0 0x7fcdd8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:51.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdeb7fe700 1 -- 172.21.15.43:0/237644505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec1596a0 msgr2=0x7fcdd8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:51.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdeb7fe700 1 --2- 172.21.15.43:0/237644505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec1596a0 0x7fcdd8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:51.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdeb7fe700 1 -- 172.21.15.43:0/237644505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdd8004d90 msgr2=0x7fcdd8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:51.994 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdeb7fe700 1 --2- 172.21.15.43:0/237644505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdd8004d90 0x7fcdd8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:51.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdeb7fe700 1 -- 172.21.15.43:0/237644505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd80035b0 con 0x7fcdd8003f40 2023-06-28T20:08:51.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdf0fe0700 1 --2- 172.21.15.43:0/237644505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdd8004d90 0x7fcdd8005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:51.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.982+0000 7fcdebfff700 1 --2- 172.21.15.43:0/237644505 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec1596a0 0x7fcdd8003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:51.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.983+0000 7fcdeb7fe700 1 --2- 172.21.15.43:0/237644505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdd8003f40 0x7fcdd8004850 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0b24d0 tx=0x7fcdd8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5c2c89b1704128 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:51.995 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.983+0000 7fcdeaffd700 1 -- 172.21.15.43:0/237644505 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd400c070 con 0x7fcdd8003f40 2023-06-28T20:08:51.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.983+0000 7fcdeaffd700 1 -- 172.21.15.43:0/237644505 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd4004020 con 0x7fcdd8003f40 2023-06-28T20:08:51.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.983+0000 7fcdeaffd700 1 -- 172.21.15.43:0/237644505 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd40043d0 con 0x7fcdd8003f40 2023-06-28T20:08:51.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.983+0000 7fcdf2a43700 1 -- 172.21.15.43:0/237644505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdd8003f40 msgr2=0x7fcdd8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:51.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.983+0000 7fcdf2a43700 1 --2- 172.21.15.43:0/237644505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdd8003f40 0x7fcdd8004850 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0b24d0 tx=0x7fcdd8005880 comp rx=0 tx=0).stop 2023-06-28T20:08:51.996 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.983+0000 7fcdf0fe0700 1 -- 172.21.15.43:0/237644505 reap_dead start 2023-06-28T20:08:51.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.983+0000 7fcdf2a43700 1 -- 172.21.15.43:0/237644505 shutdown_connections 2023-06-28T20:08:51.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.983+0000 7fcdf2a43700 1 -- 172.21.15.43:0/237644505 >> 172.21.15.43:0/237644505 conn(0x7fcdec06b000 msgr2=0x7fcdec06b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:51.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.983+0000 7fcdf2a43700 1 -- 172.21.15.43:0/237644505 shutdown_connections 2023-06-28T20:08:51.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.983+0000 7fcdf2a43700 1 -- 172.21.15.43:0/237644505 wait complete. 2023-06-28T20:08:51.997 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.985+0000 7fcdf2a43700 1 Processor -- start 2023-06-28T20:08:51.998 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.985+0000 7fcdf2a43700 1 -- start start 2023-06-28T20:08:52.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.985+0000 7fcdf2a43700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdd8003f40 0x7fcdec0c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.985+0000 7fcdf2a43700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec152b80 0x7fcdec14c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.985+0000 7fcdf2a43700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdec15a6b0 0x7fcdec155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.000 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.985+0000 7fcdf2a43700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec15a220 con 0x7fcdd8003f40 2023-06-28T20:08:52.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.985+0000 7fcdf2a43700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec0c5bf0 con 0x7fcdec15a6b0 2023-06-28T20:08:52.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.985+0000 7fcdf2a43700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec0c5910 con 0x7fcdec152b80 2023-06-28T20:08:52.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.986+0000 7fcdebfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdd8003f40 0x7fcdec0c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.001 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.986+0000 7fcdebfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdd8003f40 0x7fcdec0c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55710/0 (socket says 172.21.15.43:55710) 2023-06-28T20:08:52.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.986+0000 7fcdebfff700 1 -- 172.21.15.43:0/2898024813 learned_addr learned my addr 172.21.15.43:0/2898024813 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:52.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.986+0000 7fcdf0fe0700 1 --2- 172.21.15.43:0/2898024813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdec15a6b0 0x7fcdec155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.986+0000 7fcdeb7fe700 1 --2- 172.21.15.43:0/2898024813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec152b80 0x7fcdec14c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.986+0000 7fcdebfff700 1 -- 172.21.15.43:0/2898024813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec152b80 msgr2=0x7fcdec14c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.002 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.986+0000 7fcdebfff700 1 --2- 172.21.15.43:0/2898024813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec152b80 0x7fcdec14c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.986+0000 7fcdebfff700 1 -- 172.21.15.43:0/2898024813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdec15a6b0 msgr2=0x7fcdec155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.986+0000 7fcdebfff700 1 --2- 172.21.15.43:0/2898024813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdec15a6b0 0x7fcdec155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.986+0000 7fcdebfff700 1 -- 172.21.15.43:0/2898024813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec159bd0 con 0x7fcdd8003f40 2023-06-28T20:08:52.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.986+0000 7fcdebfff700 1 --2- 172.21.15.43:0/2898024813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdd8003f40 0x7fcdec0c52a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00e810 tx=0x7fcddc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:52.003 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.986+0000 7fcde8ff9700 1 -- 172.21.15.43:0/2898024813 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc012590 con 0x7fcdd8003f40 2023-06-28T20:08:52.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.986+0000 7fcde8ff9700 1 -- 172.21.15.43:0/2898024813 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcddc0126f0 con 0x7fcdd8003f40 2023-06-28T20:08:52.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.986+0000 7fcdf2a43700 1 -- 172.21.15.43:0/2898024813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdec159eb0 con 0x7fcdd8003f40 2023-06-28T20:08:52.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.986+0000 7fcde8ff9700 1 -- 172.21.15.43:0/2898024813 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc012a80 con 0x7fcdd8003f40 2023-06-28T20:08:52.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.986+0000 7fcdf2a43700 1 -- 172.21.15.43:0/2898024813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdec14ce20 con 0x7fcdd8003f40 2023-06-28T20:08:52.004 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.987+0000 7fcdf2a43700 1 -- 172.21.15.43:0/2898024813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdec0cc3e0 con 0x7fcdd8003f40 2023-06-28T20:08:52.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.988+0000 7fcde8ff9700 1 -- 172.21.15.43:0/2898024813 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcddc012be0 con 0x7fcdd8003f40 2023-06-28T20:08:52.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.988+0000 7fcde8ff9700 1 --2- 172.21.15.43:0/2898024813 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fcdcc06ace0 0x7fcdcc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.988+0000 7fcde8ff9700 1 -- 172.21.15.43:0/2898024813 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fcddc088ea0 con 0x7fcdd8003f40 2023-06-28T20:08:52.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.988+0000 7fcdeb7fe700 1 --2- 172.21.15.43:0/2898024813 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fcdcc06ace0 0x7fcdcc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.989+0000 7fcdeb7fe700 1 --2- 172.21.15.43:0/2898024813 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fcdcc06ace0 0x7fcdcc06d190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0b24d0 tx=0x7fcdd400c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:52.005 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:51.993+0000 7fcde8ff9700 1 -- 172.21.15.43:0/2898024813 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcddc0542e0 con 0x7fcdd8003f40 2023-06-28T20:08:52.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.192+0000 7fcdf2a43700 1 -- 172.21.15.43:0/2898024813 --> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdec15de30 con 0x7fcdcc06ace0 2023-06-28T20:08:52.195 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.192+0000 7fcde8ff9700 1 -- 172.21.15.43:0/2898024813 <== mgr.4101 v2:172.21.15.43:6832/406144153 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcdd4009560 con 0x7fcdcc06ace0 2023-06-28T20:08:52.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.194+0000 7fcdf2a43700 1 -- 172.21.15.43:0/2898024813 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fcdcc06ace0 msgr2=0x7fcdcc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.194+0000 7fcdf2a43700 1 --2- 172.21.15.43:0/2898024813 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fcdcc06ace0 0x7fcdcc06d190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcdec0b24d0 tx=0x7fcdd400c000 comp rx=0 tx=0).stop 2023-06-28T20:08:52.197 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.194+0000 7fcdf2a43700 1 -- 172.21.15.43:0/2898024813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdd8003f40 msgr2=0x7fcdec0c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.194+0000 7fcdf2a43700 1 --2- 172.21.15.43:0/2898024813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdd8003f40 0x7fcdec0c52a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00e810 tx=0x7fcddc00eb20 comp rx=0 tx=0).stop 2023-06-28T20:08:52.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.194+0000 7fcdf2a43700 1 -- 172.21.15.43:0/2898024813 shutdown_connections 2023-06-28T20:08:52.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.194+0000 7fcdf2a43700 1 --2- 172.21.15.43:0/2898024813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcdec15a6b0 0x7fcdec155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.194+0000 7fcdf2a43700 1 --2- 172.21.15.43:0/2898024813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdd8003f40 0x7fcdec0c52a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.198 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.194+0000 7fcdf2a43700 1 --2- 172.21.15.43:0/2898024813 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fcdcc06ace0 0x7fcdcc06d190 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.194+0000 7fcdf2a43700 1 --2- 172.21.15.43:0/2898024813 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fcdec152b80 0x7fcdec14c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.194+0000 7fcdf2a43700 1 -- 172.21.15.43:0/2898024813 >> 172.21.15.43:0/2898024813 conn(0x7fcdec06b000 msgr2=0x7fcdec154cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:52.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.194+0000 7fcdf2a43700 1 -- 172.21.15.43:0/2898024813 shutdown_connections 2023-06-28T20:08:52.199 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.194+0000 7fcdf2a43700 1 -- 172.21.15.43:0/2898024813 wait complete. 2023-06-28T20:08:52.200 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 0 to deep-scrub 2023-06-28T20:08:52.209 INFO:tasks.ceph:Scrubbing osd.1 2023-06-28T20:08:52.209 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-06-28T20:08:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.335+0000 7ff3ad87a700 1 Processor -- start 2023-06-28T20:08:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.335+0000 7ff3ad87a700 1 -- start start 2023-06-28T20:08:52.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.336+0000 7ff3ad87a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3a8159700 0x7ff3a80ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.336+0000 7ff3ad87a700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7ff3a80cc450 0x7ff3a8069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.342 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.336+0000 7ff3ad87a700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3a806fce0 con 0x7ff3a806f870 2023-06-28T20:08:52.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.336+0000 7ff3ad87a700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a80aeab0 con 0x7ff3a80cc450 2023-06-28T20:08:52.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.336+0000 7ff3ad87a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a8159ad0 con 0x7ff3a8159700 2023-06-28T20:08:52.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.336+0000 7ff3a67fc700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff3a806f870 0x7ff3a80cbd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44904/0 (socket says 172.21.15.43:44904) 2023-06-28T20:08:52.343 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.336+0000 7ff3a67fc700 1 -- 172.21.15.43:0/4162044314 learned_addr learned my addr 172.21.15.43:0/4162044314 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.336+0000 7ff3a6ffd700 1 --2- 172.21.15.43:0/4162044314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3a8159700 0x7ff3a80ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.336+0000 7ff3a77fe700 1 --2- 172.21.15.43:0/4162044314 >> v2:172.21.15.178:3300/0 conn(0x7ff3a80cc450 0x7ff3a8069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.336+0000 7ff3a5ffb700 1 -- 172.21.15.43:0/4162044314 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 625898046 0 0) 0x7ff3a806fce0 con 0x7ff3a806f870 2023-06-28T20:08:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.336+0000 7ff3a5ffb700 1 -- 172.21.15.43:0/4162044314 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3900035b0 con 0x7ff3a806f870 2023-06-28T20:08:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.337+0000 7ff3a5ffb700 1 -- 172.21.15.43:0/4162044314 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2789032644 0 0) 0x7ff3900035b0 con 0x7ff3a806f870 2023-06-28T20:08:52.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.337+0000 7ff3a5ffb700 1 -- 172.21.15.43:0/4162044314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3a8159700 msgr2=0x7ff3a80ae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:52.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.337+0000 7ff3a5ffb700 1 --2- 172.21.15.43:0/4162044314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3a8159700 0x7ff3a80ae1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.345 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.337+0000 7ff3a5ffb700 1 -- 172.21.15.43:0/4162044314 >> v2:172.21.15.178:3300/0 conn(0x7ff3a80cc450 msgr2=0x7ff3a8069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:52.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.337+0000 7ff3a5ffb700 1 --2- 172.21.15.43:0/4162044314 >> v2:172.21.15.178:3300/0 conn(0x7ff3a80cc450 0x7ff3a8069ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.337+0000 7ff3a5ffb700 1 -- 172.21.15.43:0/4162044314 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a80b0760 con 0x7ff3a806f870 2023-06-28T20:08:52.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.337+0000 7ff3a5ffb700 1 -- 172.21.15.43:0/4162044314 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7ff398001670 con 0x7ff3a806f870 2023-06-28T20:08:52.346 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.337+0000 7ff3a5ffb700 1 -- 172.21.15.43:0/4162044314 >> v1:172.21.15.43:6789/0 conn(0x7ff3a806f870 legacy=0x7ff3a80cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.337+0000 7ff3a5ffb700 1 --2- 172.21.15.43:0/4162044314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3a8159700 0x7ff390003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.337+0000 7ff3a5ffb700 1 --2- 172.21.15.43:0/4162044314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff390003f80 0x7ff390004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.337+0000 7ff3a5ffb700 1 --2- 172.21.15.43:0/4162044314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff390004d90 0x7ff390005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.347 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.337+0000 7ff3a5ffb700 1 -- 172.21.15.43:0/4162044314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a80aeab0 con 0x7ff390003f80 2023-06-28T20:08:52.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.337+0000 7ff3a5ffb700 1 -- 172.21.15.43:0/4162044314 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a8159ad0 con 0x7ff390004d90 2023-06-28T20:08:52.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.337+0000 7ff3a5ffb700 1 -- 172.21.15.43:0/4162044314 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff39000ba40 con 0x7ff3a8159700 2023-06-28T20:08:52.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.337+0000 7ff3a67fc700 1 --2- 172.21.15.43:0/4162044314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff390003f80 0x7ff390004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.337+0000 7ff3a6ffd700 1 --2- 172.21.15.43:0/4162044314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3a8159700 0x7ff390003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.348 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.337+0000 7ff3a77fe700 1 --2- 172.21.15.43:0/4162044314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff390004d90 0x7ff390005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.337+0000 7ff3a67fc700 1 -- 172.21.15.43:0/4162044314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3a8159700 msgr2=0x7ff390003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:52.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.337+0000 7ff3a67fc700 1 --2- 172.21.15.43:0/4162044314 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3a8159700 0x7ff390003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.337+0000 7ff3a67fc700 1 -- 172.21.15.43:0/4162044314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff390004d90 msgr2=0x7ff390005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:52.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.337+0000 7ff3a67fc700 1 --2- 172.21.15.43:0/4162044314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff390004d90 0x7ff390005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.349 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.337+0000 7ff3a67fc700 1 -- 172.21.15.43:0/4162044314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3900035b0 con 0x7ff390003f80 2023-06-28T20:08:52.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.337+0000 7ff3a67fc700 1 --2- 172.21.15.43:0/4162044314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff390003f80 0x7ff390004850 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff39800c370 tx=0x7ff390005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d195764412355b server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:52.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.338+0000 7ff3a5ffb700 1 -- 172.21.15.43:0/4162044314 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff398007c60 con 0x7ff390003f80 2023-06-28T20:08:52.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.338+0000 7ff3a5ffb700 1 -- 172.21.15.43:0/4162044314 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39800ce50 con 0x7ff390003f80 2023-06-28T20:08:52.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.338+0000 7ff3a5ffb700 1 -- 172.21.15.43:0/4162044314 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff398005ba0 con 0x7ff390003f80 2023-06-28T20:08:52.350 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.338+0000 7ff3ad87a700 1 -- 172.21.15.43:0/4162044314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff390003f80 msgr2=0x7ff390004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.338+0000 7ff3ad87a700 1 --2- 172.21.15.43:0/4162044314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff390003f80 0x7ff390004850 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff39800c370 tx=0x7ff390005880 comp rx=0 tx=0).stop 2023-06-28T20:08:52.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.338+0000 7ff3a77fe700 1 -- 172.21.15.43:0/4162044314 reap_dead start 2023-06-28T20:08:52.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.338+0000 7ff3ad87a700 1 -- 172.21.15.43:0/4162044314 shutdown_connections 2023-06-28T20:08:52.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.338+0000 7ff3ad87a700 1 -- 172.21.15.43:0/4162044314 >> 172.21.15.43:0/4162044314 conn(0x7ff3a80b2ed0 msgr2=0x7ff3a8071460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:52.351 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.338+0000 7ff3ad87a700 1 -- 172.21.15.43:0/4162044314 shutdown_connections 2023-06-28T20:08:52.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.338+0000 7ff3ad87a700 1 -- 172.21.15.43:0/4162044314 wait complete. 2023-06-28T20:08:52.352 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.339+0000 7ff3ad87a700 1 Processor -- start 2023-06-28T20:08:52.354 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.340+0000 7ff3ad87a700 1 -- start start 2023-06-28T20:08:52.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.340+0000 7ff3ad87a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff390003f80 0x7ff3a8158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.340+0000 7ff3ad87a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3a81ee770 0x7ff3a809ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.355 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.340+0000 7ff3ad87a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3a80c5020 0x7ff3a809b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.340+0000 7ff3ad87a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a81589d0 con 0x7ff3a81ee770 2023-06-28T20:08:52.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.340+0000 7ff3ad87a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a80c2c80 con 0x7ff390003f80 2023-06-28T20:08:52.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.340+0000 7ff3ad87a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a80c2f60 con 0x7ff3a80c5020 2023-06-28T20:08:52.356 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.340+0000 7ff3a67fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3a81ee770 0x7ff3a809ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.340+0000 7ff3a67fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3a81ee770 0x7ff3a809ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55732/0 (socket says 172.21.15.43:55732) 2023-06-28T20:08:52.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.340+0000 7ff3a67fc700 1 -- 172.21.15.43:0/3346461351 learned_addr learned my addr 172.21.15.43:0/3346461351 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:52.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.340+0000 7ff3a6ffd700 1 --2- 172.21.15.43:0/3346461351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff390003f80 0x7ff3a8158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.341+0000 7ff3a77fe700 1 --2- 172.21.15.43:0/3346461351 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3a80c5020 0x7ff3a809b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.357 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.341+0000 7ff3a67fc700 1 -- 172.21.15.43:0/3346461351 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3a80c5020 msgr2=0x7ff3a809b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.341+0000 7ff3a67fc700 1 --2- 172.21.15.43:0/3346461351 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ff3a80c5020 0x7ff3a809b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.341+0000 7ff3a67fc700 1 -- 172.21.15.43:0/3346461351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff390003f80 msgr2=0x7ff3a8158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.341+0000 7ff3a67fc700 1 --2- 172.21.15.43:0/3346461351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff390003f80 0x7ff3a8158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.341+0000 7ff3a67fc700 1 -- 172.21.15.43:0/3346461351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a80c3240 con 0x7ff3a81ee770 2023-06-28T20:08:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.341+0000 7ff3a67fc700 1 --2- 172.21.15.43:0/3346461351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3a81ee770 0x7ff3a809ad30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff398000c00 tx=0x7ff398000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.341+0000 7ff38ffff700 1 -- 172.21.15.43:0/3346461351 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff398020070 con 0x7ff3a81ee770 2023-06-28T20:08:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.341+0000 7ff38ffff700 1 -- 172.21.15.43:0/3346461351 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39800cc50 con 0x7ff3a81ee770 2023-06-28T20:08:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.341+0000 7ff38ffff700 1 -- 172.21.15.43:0/3346461351 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff398007310 con 0x7ff3a81ee770 2023-06-28T20:08:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.341+0000 7ff3ad87a700 1 -- 172.21.15.43:0/3346461351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a80c3520 con 0x7ff3a81ee770 2023-06-28T20:08:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.341+0000 7ff3ad87a700 1 -- 172.21.15.43:0/3346461351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a809d020 con 0x7ff3a81ee770 2023-06-28T20:08:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.342+0000 7ff3ad87a700 1 -- 172.21.15.43:0/3346461351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff3a8071cf0 con 0x7ff3a81ee770 2023-06-28T20:08:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.343+0000 7ff38ffff700 1 -- 172.21.15.43:0/3346461351 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff398007470 con 0x7ff3a81ee770 2023-06-28T20:08:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.343+0000 7ff38ffff700 1 --2- 172.21.15.43:0/3346461351 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff39406ace0 0x7ff39406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.343+0000 7ff38ffff700 1 -- 172.21.15.43:0/3346461351 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff39808b070 con 0x7ff3a81ee770 2023-06-28T20:08:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.343+0000 7ff3a6ffd700 1 --2- 172.21.15.43:0/3346461351 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff39406ace0 0x7ff39406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.343+0000 7ff38ffff700 1 --2- 172.21.15.43:0/3346461351 >> [v2:172.21.15.43:6808/894763335,v1:172.21.15.43:6809/894763335] conn(0x7ff394071150 0x7ff394073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.343+0000 7ff38ffff700 1 -- 172.21.15.43:0/3346461351 --> [v2:172.21.15.43:6808/894763335,v1:172.21.15.43:6809/894763335] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff394073c30 con 0x7ff394071150 2023-06-28T20:08:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.343+0000 7ff38ffff700 1 -- 172.21.15.43:0/3346461351 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff39808d710 con 0x7ff3a81ee770 2023-06-28T20:08:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.343+0000 7ff3a77fe700 1 --2- 172.21.15.43:0/3346461351 >> [v2:172.21.15.43:6808/894763335,v1:172.21.15.43:6809/894763335] conn(0x7ff394071150 0x7ff394073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.344+0000 7ff3a6ffd700 1 --2- 172.21.15.43:0/3346461351 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff39406ace0 0x7ff39406d190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009b80 tx=0x7ff39c009210 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:52.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.344+0000 7ff3a77fe700 1 --2- 172.21.15.43:0/3346461351 >> [v2:172.21.15.43:6808/894763335,v1:172.21.15.43:6809/894763335] conn(0x7ff394071150 0x7ff394073560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:52.362 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.344+0000 7ff38ffff700 1 -- 172.21.15.43:0/3346461351 <== osd.1 v2:172.21.15.43:6808/894763335 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff394073c30 con 0x7ff394071150 2023-06-28T20:08:52.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.359+0000 7ff3ad87a700 1 -- 172.21.15.43:0/3346461351 --> [v2:172.21.15.43:6808/894763335,v1:172.21.15.43:6809/894763335] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff3a80b0a40 con 0x7ff394071150 2023-06-28T20:08:52.363 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.359+0000 7ff38ffff700 1 -- 172.21.15.43:0/3346461351 <== osd.1 v2:172.21.15.43:6808/894763335 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff3a000f040 con 0x7ff394071150 2023-06-28T20:08:52.363 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-06-28T20:08:52.364 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-28T20:08:52.364 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-06-28T20:08:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.359+0000 7ff3ad87a700 1 -- 172.21.15.43:0/3346461351 >> [v2:172.21.15.43:6808/894763335,v1:172.21.15.43:6809/894763335] conn(0x7ff394071150 msgr2=0x7ff394073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.359+0000 7ff3ad87a700 1 --2- 172.21.15.43:0/3346461351 >> [v2:172.21.15.43:6808/894763335,v1:172.21.15.43:6809/894763335] conn(0x7ff394071150 0x7ff394073560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.359+0000 7ff3ad87a700 1 -- 172.21.15.43:0/3346461351 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff39406ace0 msgr2=0x7ff39406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.359+0000 7ff3ad87a700 1 --2- 172.21.15.43:0/3346461351 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ff39406ace0 0x7ff39406d190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff39c009b80 tx=0x7ff39c009210 comp rx=0 tx=0).stop 2023-06-28T20:08:52.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.359+0000 7ff3ad87a700 1 -- 172.21.15.43:0/3346461351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3a81ee770 msgr2=0x7ff3a809ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.359+0000 7ff3ad87a700 1 --2- 172.21.15.43:0/3346461351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3a81ee770 0x7ff3a809ad30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff398000c00 tx=0x7ff398000fe0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.359+0000 7ff3a77fe700 1 -- 172.21.15.43:0/3346461351 reap_dead start 2023-06-28T20:08:52.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.359+0000 7ff3ad87a700 1 -- 172.21.15.43:0/3346461351 shutdown_connections 2023-06-28T20:08:52.366 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.359+0000 7ff3ad87a700 1 -- 172.21.15.43:0/3346461351 >> 172.21.15.43:0/3346461351 conn(0x7ff3a80b2ed0 msgr2=0x7ff3a80ad340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:52.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.359+0000 7ff3ad87a700 1 -- 172.21.15.43:0/3346461351 shutdown_connections 2023-06-28T20:08:52.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.359+0000 7ff3ad87a700 1 -- 172.21.15.43:0/3346461351 wait complete. 2023-06-28T20:08:52.374 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-06-28T20:08:52.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.497+0000 7f8959e52700 1 Processor -- start 2023-06-28T20:08:52.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.497+0000 7f8959e52700 1 -- start start 2023-06-28T20:08:52.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.498+0000 7f8959e52700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f89540cfe20 0x7f895415d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.498+0000 7f8959e52700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f895415cce0 con 0x7f895415d990 2023-06-28T20:08:52.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.498+0000 7f8959e52700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89540ae770 con 0x7f895415a180 2023-06-28T20:08:52.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.498+0000 7f8959e52700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89540ad0a0 con 0x7f89540cfe20 2023-06-28T20:08:52.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.498+0000 7f8953fff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f895415d990 0x7f895406c0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44930/0 (socket says 172.21.15.43:44930) 2023-06-28T20:08:52.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.498+0000 7f8953fff700 1 -- 172.21.15.43:0/679204603 learned_addr learned my addr 172.21.15.43:0/679204603 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:52.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.498+0000 7f8952ffd700 1 --2- 172.21.15.43:0/679204603 >> v2:172.21.15.178:3300/0 conn(0x7f89540cfe20 0x7f895415d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.498+0000 7f89527fc700 1 -- 172.21.15.43:0/679204603 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1330714021 0 0) 0x7f895415cce0 con 0x7f895415d990 2023-06-28T20:08:52.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.498+0000 7f89527fc700 1 -- 172.21.15.43:0/679204603 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f893c0035b0 con 0x7f895415d990 2023-06-28T20:08:52.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.498+0000 7f89527fc700 1 -- 172.21.15.43:0/679204603 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1264872166 0 0) 0x7f893c0035b0 con 0x7f895415d990 2023-06-28T20:08:52.504 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.498+0000 7f8952ffd700 1 -- 172.21.15.43:0/679204603 >> v1:172.21.15.178:6789/0 conn(0x7f895415a180 legacy=0x7f89540ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.498+0000 7f8952ffd700 1 -- 172.21.15.43:0/679204603 >> v1:172.21.15.43:6789/0 conn(0x7f895415d990 legacy=0x7f895406c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.498+0000 7f8952ffd700 1 -- 172.21.15.43:0/679204603 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89540ad380 con 0x7f89540cfe20 2023-06-28T20:08:52.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.499+0000 7f8952ffd700 1 --2- 172.21.15.43:0/679204603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89540cfe20 0x7f895415d450 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8948009840 tx=0x7f8948009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e409253498b25ab8 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:52.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.499+0000 7f89527fc700 1 -- 172.21.15.43:0/679204603 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c0019f0 con 0x7f89540cfe20 2023-06-28T20:08:52.505 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.499+0000 7f89527fc700 1 -- 172.21.15.43:0/679204603 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8948004030 con 0x7f89540cfe20 2023-06-28T20:08:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.499+0000 7f89527fc700 1 -- 172.21.15.43:0/679204603 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89480043c0 con 0x7f89540cfe20 2023-06-28T20:08:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.499+0000 7f8959e52700 1 -- 172.21.15.43:0/679204603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89540cfe20 msgr2=0x7f895415d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.499+0000 7f8959e52700 1 --2- 172.21.15.43:0/679204603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89540cfe20 0x7f895415d450 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8948009840 tx=0x7f8948009b50 comp rx=0 tx=0).stop 2023-06-28T20:08:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.499+0000 7f8959e52700 1 -- 172.21.15.43:0/679204603 shutdown_connections 2023-06-28T20:08:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.499+0000 7f8959e52700 1 --2- 172.21.15.43:0/679204603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89540cfe20 0x7f895415d450 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.499+0000 7f8959e52700 1 -- 172.21.15.43:0/679204603 >> 172.21.15.43:0/679204603 conn(0x7f89540b1a50 msgr2=0x7f89540cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:52.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.499+0000 7f8959e52700 1 -- 172.21.15.43:0/679204603 shutdown_connections 2023-06-28T20:08:52.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.499+0000 7f8959e52700 1 -- 172.21.15.43:0/679204603 wait complete. 2023-06-28T20:08:52.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.501+0000 7f8959e52700 1 Processor -- start 2023-06-28T20:08:52.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.501+0000 7f8959e52700 1 -- start start 2023-06-28T20:08:52.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.501+0000 7f8959e52700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89540cfe20 0x7f8954149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.501+0000 7f8959e52700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f895415a180 0x7f895415be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.501+0000 7f8959e52700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895415c3b0 0x7f8954157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.502+0000 7f8959e52700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895415c830 con 0x7f895415c3b0 2023-06-28T20:08:52.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.502+0000 7f8959e52700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89541589a0 con 0x7f895415a180 2023-06-28T20:08:52.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.502+0000 7f8959e52700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89540b3960 con 0x7f89540cfe20 2023-06-28T20:08:52.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.502+0000 7f8953fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895415c3b0 0x7f8954157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.502+0000 7f8953fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895415c3b0 0x7f8954157c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55750/0 (socket says 172.21.15.43:55750) 2023-06-28T20:08:52.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.502+0000 7f8953fff700 1 -- 172.21.15.43:0/2856392508 learned_addr learned my addr 172.21.15.43:0/2856392508 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:52.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.502+0000 7f89537fe700 1 --2- 172.21.15.43:0/2856392508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89540cfe20 0x7f8954149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.502+0000 7f8952ffd700 1 --2- 172.21.15.43:0/2856392508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f895415a180 0x7f895415be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.502+0000 7f8953fff700 1 -- 172.21.15.43:0/2856392508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89540cfe20 msgr2=0x7f8954149aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.502+0000 7f8953fff700 1 --2- 172.21.15.43:0/2856392508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89540cfe20 0x7f8954149aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.502+0000 7f8953fff700 1 -- 172.21.15.43:0/2856392508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f895415a180 msgr2=0x7f895415be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.502+0000 7f8953fff700 1 --2- 172.21.15.43:0/2856392508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f895415a180 0x7f895415be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.502+0000 7f8953fff700 1 -- 172.21.15.43:0/2856392508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89480094a0 con 0x7f895415c3b0 2023-06-28T20:08:52.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.502+0000 7f8953fff700 1 --2- 172.21.15.43:0/2856392508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895415c3b0 0x7f8954157c70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f894c001890 tx=0x7f894c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:52.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.502+0000 7f893bfff700 1 -- 172.21.15.43:0/2856392508 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c00acb0 con 0x7f895415c3b0 2023-06-28T20:08:52.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.502+0000 7f893bfff700 1 -- 172.21.15.43:0/2856392508 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f894c0019e0 con 0x7f895415c3b0 2023-06-28T20:08:52.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.502+0000 7f8959e52700 1 -- 172.21.15.43:0/2856392508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89540b3c00 con 0x7f895415c3b0 2023-06-28T20:08:52.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.502+0000 7f893bfff700 1 -- 172.21.15.43:0/2856392508 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f894c0072d0 con 0x7f895415c3b0 2023-06-28T20:08:52.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.502+0000 7f8959e52700 1 -- 172.21.15.43:0/2856392508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89540b41c0 con 0x7f895415c3b0 2023-06-28T20:08:52.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.503+0000 7f8959e52700 1 -- 172.21.15.43:0/2856392508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89540c4a70 con 0x7f895415c3b0 2023-06-28T20:08:52.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.504+0000 7f893bfff700 1 -- 172.21.15.43:0/2856392508 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f894c007430 con 0x7f895415c3b0 2023-06-28T20:08:52.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.504+0000 7f893bfff700 1 --2- 172.21.15.43:0/2856392508 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f894006ace0 0x7f894006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.504+0000 7f893bfff700 1 -- 172.21.15.43:0/2856392508 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f894c089100 con 0x7f895415c3b0 2023-06-28T20:08:52.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.504+0000 7f89537fe700 1 --2- 172.21.15.43:0/2856392508 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f894006ace0 0x7f894006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.505+0000 7f89537fe700 1 --2- 172.21.15.43:0/2856392508 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f894006ace0 0x7f894006d190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8944000f60 tx=0x7f8944009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:52.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.509+0000 7f893bfff700 1 -- 172.21.15.43:0/2856392508 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f894c0544b0 con 0x7f895415c3b0 2023-06-28T20:08:52.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.707+0000 7f8959e52700 1 -- 172.21.15.43:0/2856392508 --> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f895415a550 con 0x7f894006ace0 2023-06-28T20:08:52.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.707+0000 7f893bfff700 1 -- 172.21.15.43:0/2856392508 <== mgr.4101 v2:172.21.15.43:6832/406144153 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8944001570 con 0x7f894006ace0 2023-06-28T20:08:52.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.709+0000 7f8959e52700 1 -- 172.21.15.43:0/2856392508 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f894006ace0 msgr2=0x7f894006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.709+0000 7f8959e52700 1 --2- 172.21.15.43:0/2856392508 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f894006ace0 0x7f894006d190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8944000f60 tx=0x7f8944009040 comp rx=0 tx=0).stop 2023-06-28T20:08:52.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.709+0000 7f8959e52700 1 -- 172.21.15.43:0/2856392508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895415c3b0 msgr2=0x7f8954157c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.709+0000 7f8959e52700 1 --2- 172.21.15.43:0/2856392508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895415c3b0 0x7f8954157c70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f894c001890 tx=0x7f894c00a340 comp rx=0 tx=0).stop 2023-06-28T20:08:52.714 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.709+0000 7f8959e52700 1 -- 172.21.15.43:0/2856392508 shutdown_connections 2023-06-28T20:08:52.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.709+0000 7f8959e52700 1 --2- 172.21.15.43:0/2856392508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f895415c3b0 0x7f8954157c70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.709+0000 7f8959e52700 1 --2- 172.21.15.43:0/2856392508 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f894006ace0 0x7f894006d190 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.709+0000 7f8959e52700 1 --2- 172.21.15.43:0/2856392508 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f89540cfe20 0x7f8954149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.715 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.709+0000 7f8959e52700 1 --2- 172.21.15.43:0/2856392508 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f895415a180 0x7f895415be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.709+0000 7f8959e52700 1 -- 172.21.15.43:0/2856392508 >> 172.21.15.43:0/2856392508 conn(0x7f89540b1a50 msgr2=0x7f89540ac870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:52.716 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.709+0000 7f8959e52700 1 -- 172.21.15.43:0/2856392508 shutdown_connections 2023-06-28T20:08:52.717 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.709+0000 7f8959e52700 1 -- 172.21.15.43:0/2856392508 wait complete. 2023-06-28T20:08:52.717 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 1 to deep-scrub 2023-06-28T20:08:52.724 INFO:tasks.ceph:Scrubbing osd.2 2023-06-28T20:08:52.725 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-06-28T20:08:52.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.850+0000 7fde20b37700 1 Processor -- start 2023-06-28T20:08:52.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.850+0000 7fde20b37700 1 -- start start 2023-06-28T20:08:52.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.851+0000 7fde20b37700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fde1c0b4f20 0x7fde1c15d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.851+0000 7fde20b37700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde1c0d0990 con 0x7fde1c0d0e20 2023-06-28T20:08:52.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.851+0000 7fde20b37700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde1c0d19b0 con 0x7fde1c15ceb0 2023-06-28T20:08:52.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.851+0000 7fde20b37700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1c0b8960 con 0x7fde1c0b4f20 2023-06-28T20:08:52.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.851+0000 7fde19d9b700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fde1c0b4f20 0x7fde1c15d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.851+0000 7fde1ad9d700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fde1c0d0e20 0x7fde1c0d12a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44944/0 (socket says 172.21.15.43:44944) 2023-06-28T20:08:52.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.851+0000 7fde1ad9d700 1 -- 172.21.15.43:0/3172372911 learned_addr learned my addr 172.21.15.43:0/3172372911 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:52.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.852+0000 7fde19d9b700 1 -- 172.21.15.43:0/3172372911 >> v1:172.21.15.178:6789/0 conn(0x7fde1c15ceb0 legacy=0x7fde1c0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.852+0000 7fde19d9b700 1 -- 172.21.15.43:0/3172372911 >> v1:172.21.15.43:6789/0 conn(0x7fde1c0d0e20 legacy=0x7fde1c0d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.852+0000 7fde19d9b700 1 -- 172.21.15.43:0/3172372911 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c15d990 con 0x7fde1c0b4f20 2023-06-28T20:08:52.858 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.852+0000 7fde1959a700 1 -- 172.21.15.43:0/3172372911 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277102135 0 0) 0x7fde1c0d0990 con 0x7fde1c0d0e20 2023-06-28T20:08:52.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.852+0000 7fde19d9b700 1 --2- 172.21.15.43:0/3172372911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c0b4f20 0x7fde1c15d280 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fde0c009840 tx=0x7fde0c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770150d592c06087 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:52.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.852+0000 7fde1959a700 1 -- 172.21.15.43:0/3172372911 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c011830 con 0x7fde1c0b4f20 2023-06-28T20:08:52.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.852+0000 7fde1959a700 1 -- 172.21.15.43:0/3172372911 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0c011990 con 0x7fde1c0b4f20 2023-06-28T20:08:52.859 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.852+0000 7fde1959a700 1 -- 172.21.15.43:0/3172372911 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c011d20 con 0x7fde1c0b4f20 2023-06-28T20:08:52.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.853+0000 7fde20b37700 1 -- 172.21.15.43:0/3172372911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c0b4f20 msgr2=0x7fde1c15d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.853+0000 7fde20b37700 1 --2- 172.21.15.43:0/3172372911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c0b4f20 0x7fde1c15d280 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fde0c009840 tx=0x7fde0c009b50 comp rx=0 tx=0).stop 2023-06-28T20:08:52.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.853+0000 7fde20b37700 1 -- 172.21.15.43:0/3172372911 shutdown_connections 2023-06-28T20:08:52.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.853+0000 7fde20b37700 1 --2- 172.21.15.43:0/3172372911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c0b4f20 0x7fde1c15d280 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.853+0000 7fde20b37700 1 -- 172.21.15.43:0/3172372911 >> 172.21.15.43:0/3172372911 conn(0x7fde1c06a5c0 msgr2=0x7fde1c06a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:52.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.853+0000 7fde20b37700 1 -- 172.21.15.43:0/3172372911 shutdown_connections 2023-06-28T20:08:52.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.853+0000 7fde20b37700 1 -- 172.21.15.43:0/3172372911 wait complete. 2023-06-28T20:08:52.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.854+0000 7fde20b37700 1 Processor -- start 2023-06-28T20:08:52.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.854+0000 7fde20b37700 1 -- start start 2023-06-28T20:08:52.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.855+0000 7fde20b37700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde1c0b4f20 0x7fde1c156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.855+0000 7fde20b37700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c0d0e20 0x7fde1c156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.855+0000 7fde20b37700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde1c15ceb0 0x7fde1c0b2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.855+0000 7fde20b37700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c0cbd10 con 0x7fde1c0d0e20 2023-06-28T20:08:52.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.855+0000 7fde20b37700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c157440 con 0x7fde1c15ceb0 2023-06-28T20:08:52.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.855+0000 7fde20b37700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c0b2100 con 0x7fde1c0b4f20 2023-06-28T20:08:52.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.855+0000 7fde19d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c0d0e20 0x7fde1c156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.855+0000 7fde19d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c0d0e20 0x7fde1c156f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55762/0 (socket says 172.21.15.43:55762) 2023-06-28T20:08:52.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.855+0000 7fde19d9b700 1 -- 172.21.15.43:0/4210090063 learned_addr learned my addr 172.21.15.43:0/4210090063 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:52.868 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.855+0000 7fde1ad9d700 1 --2- 172.21.15.43:0/4210090063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde1c15ceb0 0x7fde1c0b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.855+0000 7fde1a59c700 1 --2- 172.21.15.43:0/4210090063 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde1c0b4f20 0x7fde1c156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.855+0000 7fde19d9b700 1 -- 172.21.15.43:0/4210090063 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde1c0b4f20 msgr2=0x7fde1c156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.869 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.855+0000 7fde19d9b700 1 --2- 172.21.15.43:0/4210090063 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde1c0b4f20 0x7fde1c156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.855+0000 7fde19d9b700 1 -- 172.21.15.43:0/4210090063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde1c15ceb0 msgr2=0x7fde1c0b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.855+0000 7fde19d9b700 1 --2- 172.21.15.43:0/4210090063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde1c15ceb0 0x7fde1c0b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.855+0000 7fde19d9b700 1 -- 172.21.15.43:0/4210090063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0c0094a0 con 0x7fde1c0d0e20 2023-06-28T20:08:52.870 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.855+0000 7fde1a59c700 1 --2- 172.21.15.43:0/4210090063 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fde1c0b4f20 0x7fde1c156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:52.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.856+0000 7fde19d9b700 1 --2- 172.21.15.43:0/4210090063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c0d0e20 0x7fde1c156f00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0090b0 tx=0x7fde0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:52.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.856+0000 7fde12ffd700 1 -- 172.21.15.43:0/4210090063 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c01a850 con 0x7fde1c0d0e20 2023-06-28T20:08:52.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.856+0000 7fde12ffd700 1 -- 172.21.15.43:0/4210090063 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0c01a9b0 con 0x7fde1c0d0e20 2023-06-28T20:08:52.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.856+0000 7fde20b37700 1 -- 172.21.15.43:0/4210090063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1c0cb370 con 0x7fde1c0d0e20 2023-06-28T20:08:52.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.856+0000 7fde12ffd700 1 -- 172.21.15.43:0/4210090063 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c01ad20 con 0x7fde1c0d0e20 2023-06-28T20:08:52.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.856+0000 7fde20b37700 1 -- 172.21.15.43:0/4210090063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1c0cb930 con 0x7fde1c0d0e20 2023-06-28T20:08:52.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.856+0000 7fde20b37700 1 -- 172.21.15.43:0/4210090063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fde1c0cd900 con 0x7fde1c0d0e20 2023-06-28T20:08:52.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.857+0000 7fde12ffd700 1 -- 172.21.15.43:0/4210090063 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fde0c022070 con 0x7fde1c0d0e20 2023-06-28T20:08:52.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.857+0000 7fde12ffd700 1 --2- 172.21.15.43:0/4210090063 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fddfc06ace0 0x7fddfc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.872 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.858+0000 7fde12ffd700 1 -- 172.21.15.43:0/4210090063 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fde0c08a320 con 0x7fde1c0d0e20 2023-06-28T20:08:52.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.858+0000 7fde1a59c700 1 --2- 172.21.15.43:0/4210090063 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fddfc06ace0 0x7fddfc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.858+0000 7fde12ffd700 1 --2- 172.21.15.43:0/4210090063 >> [v2:172.21.15.43:6824/1523179953,v1:172.21.15.43:6825/1523179953] conn(0x7fddfc071150 0x7fddfc073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:52.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.858+0000 7fde12ffd700 1 -- 172.21.15.43:0/4210090063 --> [v2:172.21.15.43:6824/1523179953,v1:172.21.15.43:6825/1523179953] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fddfc073c30 con 0x7fddfc071150 2023-06-28T20:08:52.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.858+0000 7fde12ffd700 1 -- 172.21.15.43:0/4210090063 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde0c08c960 con 0x7fde1c0d0e20 2023-06-28T20:08:52.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.858+0000 7fde1ad9d700 1 --2- 172.21.15.43:0/4210090063 >> [v2:172.21.15.43:6824/1523179953,v1:172.21.15.43:6825/1523179953] conn(0x7fddfc071150 0x7fddfc073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:52.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.858+0000 7fde1a59c700 1 --2- 172.21.15.43:0/4210090063 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fddfc06ace0 0x7fddfc06d190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fde04000f60 tx=0x7fde04009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:52.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.858+0000 7fde1ad9d700 1 --2- 172.21.15.43:0/4210090063 >> [v2:172.21.15.43:6824/1523179953,v1:172.21.15.43:6825/1523179953] conn(0x7fddfc071150 0x7fddfc073560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:52.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.859+0000 7fde12ffd700 1 -- 172.21.15.43:0/4210090063 <== osd.2 v2:172.21.15.43:6824/1523179953 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fddfc073c30 con 0x7fddfc071150 2023-06-28T20:08:52.876 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.874+0000 7fde20b37700 1 -- 172.21.15.43:0/4210090063 --> [v2:172.21.15.43:6824/1523179953,v1:172.21.15.43:6825/1523179953] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fde1c0cd620 con 0x7fddfc071150 2023-06-28T20:08:52.877 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.874+0000 7fde12ffd700 1 -- 172.21.15.43:0/4210090063 <== osd.2 v2:172.21.15.43:6824/1523179953 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fde14007240 con 0x7fddfc071150 2023-06-28T20:08:52.877 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-06-28T20:08:52.877 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-28T20:08:52.878 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-06-28T20:08:52.878 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.874+0000 7fde20b37700 1 -- 172.21.15.43:0/4210090063 >> [v2:172.21.15.43:6824/1523179953,v1:172.21.15.43:6825/1523179953] conn(0x7fddfc071150 msgr2=0x7fddfc073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.874+0000 7fde20b37700 1 --2- 172.21.15.43:0/4210090063 >> [v2:172.21.15.43:6824/1523179953,v1:172.21.15.43:6825/1523179953] conn(0x7fddfc071150 0x7fddfc073560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:52.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.874+0000 7fde20b37700 1 -- 172.21.15.43:0/4210090063 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fddfc06ace0 msgr2=0x7fddfc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.874+0000 7fde20b37700 1 --2- 172.21.15.43:0/4210090063 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fddfc06ace0 0x7fddfc06d190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fde04000f60 tx=0x7fde04009040 comp rx=0 tx=0).stop 2023-06-28T20:08:52.879 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.874+0000 7fde20b37700 1 -- 172.21.15.43:0/4210090063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c0d0e20 msgr2=0x7fde1c156f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:52.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.874+0000 7fde20b37700 1 --2- 172.21.15.43:0/4210090063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c0d0e20 0x7fde1c156f00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0090b0 tx=0x7fde0c004750 comp rx=0 tx=0).stop 2023-06-28T20:08:52.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.874+0000 7fde1ad9d700 1 -- 172.21.15.43:0/4210090063 reap_dead start 2023-06-28T20:08:52.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.874+0000 7fde20b37700 1 -- 172.21.15.43:0/4210090063 shutdown_connections 2023-06-28T20:08:52.880 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.874+0000 7fde20b37700 1 -- 172.21.15.43:0/4210090063 >> 172.21.15.43:0/4210090063 conn(0x7fde1c06a5c0 msgr2=0x7fde1c15dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:52.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.874+0000 7fde20b37700 1 -- 172.21.15.43:0/4210090063 shutdown_connections 2023-06-28T20:08:52.881 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:52.874+0000 7fde20b37700 1 -- 172.21.15.43:0/4210090063 wait complete. 2023-06-28T20:08:52.889 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-06-28T20:08:53.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.013+0000 7f5c08008700 1 Processor -- start 2023-06-28T20:08:53.016 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.013+0000 7f5c08008700 1 -- start start 2023-06-28T20:08:53.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.014+0000 7f5c08008700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c001592d0 0x7f5c000b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:53.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.014+0000 7f5c08008700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c001596a0 0x7f5c000b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:53.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.014+0000 7f5c08008700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c00070b40 con 0x7f5c00071900 2023-06-28T20:08:53.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.014+0000 7f5c08008700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c00070e20 con 0x7f5c001596a0 2023-06-28T20:08:53.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.014+0000 7f5c08008700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c00071100 con 0x7f5c001592d0 2023-06-28T20:08:53.021 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.014+0000 7f5c055a3700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c001596a0 0x7f5c000b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:53.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.014+0000 7f5c05da4700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c001592d0 0x7f5c000b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:53.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.014+0000 7f5c055a3700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c001596a0 0x7f5c000b1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55772/0 (socket says 172.21.15.43:55772) 2023-06-28T20:08:53.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.014+0000 7f5c055a3700 1 -- 172.21.15.43:0/78435519 learned_addr learned my addr 172.21.15.43:0/78435519 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:53.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.014+0000 7f5c065a5700 1 --1- 172.21.15.43:0/78435519 >> v1:172.21.15.178:6789/0 conn(0x7f5c00071900 0x7f5c0015d070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.178:6789/0 says I am v1:172.21.15.43:44084/0 (socket says 172.21.15.43:44084) 2023-06-28T20:08:53.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.014+0000 7f5c055a3700 1 -- 172.21.15.43:0/78435519 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c001592d0 msgr2=0x7f5c000b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:53.022 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.014+0000 7f5c055a3700 1 --2- 172.21.15.43:0/78435519 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c001592d0 0x7f5c000b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:53.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.014+0000 7f5c055a3700 1 -- 172.21.15.43:0/78435519 >> v1:172.21.15.178:6789/0 conn(0x7f5c00071900 legacy=0x7f5c0015d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:53.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.014+0000 7f5c055a3700 1 -- 172.21.15.43:0/78435519 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c000713e0 con 0x7f5c001596a0 2023-06-28T20:08:53.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.014+0000 7f5c04da2700 1 -- 172.21.15.43:0/78435519 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3005999914 0 0) 0x7f5c00070b40 con 0x7f5c00071900 2023-06-28T20:08:53.023 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.014+0000 7f5c055a3700 1 --2- 172.21.15.43:0/78435519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c001596a0 0x7f5c000b1f10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5bf00098f0 tx=0x7f5bf0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43942441576109b4 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:53.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.015+0000 7f5c04da2700 1 -- 172.21.15.43:0/78435519 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0011830 con 0x7f5c001596a0 2023-06-28T20:08:53.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.015+0000 7f5c04da2700 1 -- 172.21.15.43:0/78435519 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf0011990 con 0x7f5c001596a0 2023-06-28T20:08:53.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.015+0000 7f5c04da2700 1 -- 172.21.15.43:0/78435519 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0011d20 con 0x7f5c001596a0 2023-06-28T20:08:53.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.015+0000 7f5c08008700 1 -- 172.21.15.43:0/78435519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c001596a0 msgr2=0x7f5c000b1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:53.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.015+0000 7f5c08008700 1 --2- 172.21.15.43:0/78435519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c001596a0 0x7f5c000b1f10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5bf00098f0 tx=0x7f5bf0009c00 comp rx=0 tx=0).stop 2023-06-28T20:08:53.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.015+0000 7f5c08008700 1 -- 172.21.15.43:0/78435519 shutdown_connections 2023-06-28T20:08:53.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.015+0000 7f5c08008700 1 --2- 172.21.15.43:0/78435519 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c001592d0 0x7f5c000b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:53.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.015+0000 7f5c08008700 1 --2- 172.21.15.43:0/78435519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c001596a0 0x7f5c000b1f10 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:53.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.015+0000 7f5c08008700 1 -- 172.21.15.43:0/78435519 >> 172.21.15.43:0/78435519 conn(0x7f5c0006b000 msgr2=0x7f5c0006b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:53.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.015+0000 7f5c08008700 1 -- 172.21.15.43:0/78435519 shutdown_connections 2023-06-28T20:08:53.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.015+0000 7f5c08008700 1 -- 172.21.15.43:0/78435519 wait complete. 2023-06-28T20:08:53.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.017+0000 7f5c08008700 1 Processor -- start 2023-06-28T20:08:53.026 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.017+0000 7f5c08008700 1 -- start start 2023-06-28T20:08:53.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.018+0000 7f5c08008700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c00071900 0x7f5c00145ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:53.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.018+0000 7f5c08008700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c001592d0 0x7f5c00147600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:53.028 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.018+0000 7f5c08008700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c001596a0 0x7f5c000b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:53.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.018+0000 7f5c08008700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c001645b0 con 0x7f5c001592d0 2023-06-28T20:08:53.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.018+0000 7f5c08008700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c00148110 con 0x7f5c00071900 2023-06-28T20:08:53.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.018+0000 7f5c08008700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c001483f0 con 0x7f5c001596a0 2023-06-28T20:08:53.029 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.018+0000 7f5c055a3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c001592d0 0x7f5c00147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:53.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.018+0000 7f5c055a3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c001592d0 0x7f5c00147600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55778/0 (socket says 172.21.15.43:55778) 2023-06-28T20:08:53.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.018+0000 7f5c055a3700 1 -- 172.21.15.43:0/3876383547 learned_addr learned my addr 172.21.15.43:0/3876383547 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:53.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.018+0000 7f5c055a3700 1 -- 172.21.15.43:0/3876383547 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c001596a0 msgr2=0x7f5c000b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:53.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.018+0000 7f5c055a3700 1 --2- 172.21.15.43:0/3876383547 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c001596a0 0x7f5c000b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:53.030 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.018+0000 7f5c05da4700 1 --2- 172.21.15.43:0/3876383547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c00071900 0x7f5c00145ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:53.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.018+0000 7f5c055a3700 1 -- 172.21.15.43:0/3876383547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c00071900 msgr2=0x7f5c00145ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:53.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.018+0000 7f5c055a3700 1 --2- 172.21.15.43:0/3876383547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c00071900 0x7f5c00145ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:53.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.018+0000 7f5c055a3700 1 -- 172.21.15.43:0/3876383547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf0009550 con 0x7f5c001592d0 2023-06-28T20:08:53.031 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.018+0000 7f5c055a3700 1 --2- 172.21.15.43:0/3876383547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c001592d0 0x7f5c00147600 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009130 tx=0x7f5bf0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:53.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.018+0000 7f5bf67fc700 1 -- 172.21.15.43:0/3876383547 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf00036a0 con 0x7f5c001592d0 2023-06-28T20:08:53.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.018+0000 7f5bf67fc700 1 -- 172.21.15.43:0/3876383547 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf001e070 con 0x7f5c001592d0 2023-06-28T20:08:53.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.018+0000 7f5bf67fc700 1 -- 172.21.15.43:0/3876383547 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0003a20 con 0x7f5c001592d0 2023-06-28T20:08:53.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.018+0000 7f5c08008700 1 -- 172.21.15.43:0/3876383547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c001486d0 con 0x7f5c001592d0 2023-06-28T20:08:53.032 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.019+0000 7f5c08008700 1 -- 172.21.15.43:0/3876383547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c001460e0 con 0x7f5c001592d0 2023-06-28T20:08:53.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.019+0000 7f5c08008700 1 -- 172.21.15.43:0/3876383547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c000cc3e0 con 0x7f5c001592d0 2023-06-28T20:08:53.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.020+0000 7f5bf67fc700 1 -- 172.21.15.43:0/3876383547 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5bf0003b80 con 0x7f5c001592d0 2023-06-28T20:08:53.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.020+0000 7f5bf67fc700 1 --2- 172.21.15.43:0/3876383547 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5be406ace0 0x7f5be406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:53.033 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.020+0000 7f5bf67fc700 1 -- 172.21.15.43:0/3876383547 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5bf008a100 con 0x7f5c001592d0 2023-06-28T20:08:53.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.020+0000 7f5c05da4700 1 --2- 172.21.15.43:0/3876383547 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5be406ace0 0x7f5be406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:53.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.021+0000 7f5c05da4700 1 --2- 172.21.15.43:0/3876383547 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5be406ace0 0x7f5be406d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc009ab0 tx=0x7f5bfc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:53.034 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.025+0000 7f5bf67fc700 1 -- 172.21.15.43:0/3876383547 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5bf0055540 con 0x7f5c001592d0 2023-06-28T20:08:53.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.225+0000 7f5c08008700 1 -- 172.21.15.43:0/3876383547 --> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c0015de30 con 0x7f5be406ace0 2023-06-28T20:08:53.228 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.225+0000 7f5bf67fc700 1 -- 172.21.15.43:0/3876383547 <== mgr.4101 v2:172.21.15.43:6832/406144153 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5bfc0106e0 con 0x7f5be406ace0 2023-06-28T20:08:53.230 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.227+0000 7f5c08008700 1 -- 172.21.15.43:0/3876383547 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5be406ace0 msgr2=0x7f5be406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:53.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.227+0000 7f5c08008700 1 --2- 172.21.15.43:0/3876383547 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5be406ace0 0x7f5be406d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc009ab0 tx=0x7f5bfc008040 comp rx=0 tx=0).stop 2023-06-28T20:08:53.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.227+0000 7f5c08008700 1 -- 172.21.15.43:0/3876383547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c001592d0 msgr2=0x7f5c00147600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:53.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.227+0000 7f5c08008700 1 --2- 172.21.15.43:0/3876383547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c001592d0 0x7f5c00147600 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009130 tx=0x7f5bf0004750 comp rx=0 tx=0).stop 2023-06-28T20:08:53.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.227+0000 7f5c08008700 1 -- 172.21.15.43:0/3876383547 shutdown_connections 2023-06-28T20:08:53.231 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.227+0000 7f5c08008700 1 --2- 172.21.15.43:0/3876383547 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f5be406ace0 0x7f5be406d190 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:53.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.227+0000 7f5c08008700 1 --2- 172.21.15.43:0/3876383547 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f5c001596a0 0x7f5c000b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:53.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.227+0000 7f5c08008700 1 --2- 172.21.15.43:0/3876383547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c00071900 0x7f5c00145ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:53.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.227+0000 7f5c08008700 1 --2- 172.21.15.43:0/3876383547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c001592d0 0x7f5c00147600 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:53.232 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.227+0000 7f5c08008700 1 -- 172.21.15.43:0/3876383547 >> 172.21.15.43:0/3876383547 conn(0x7f5c0006b000 msgr2=0x7f5c000afa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:53.233 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.227+0000 7f5c08008700 1 -- 172.21.15.43:0/3876383547 shutdown_connections 2023-06-28T20:08:53.236 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.227+0000 7f5c08008700 1 -- 172.21.15.43:0/3876383547 wait complete. 2023-06-28T20:08:53.237 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 2 to deep-scrub 2023-06-28T20:08:53.242 INFO:tasks.ceph:Scrubbing osd.3 2023-06-28T20:08:53.243 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-06-28T20:08:53.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.368+0000 7f0f28b49700 1 Processor -- start 2023-06-28T20:08:53.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.368+0000 7f0f28b49700 1 -- start start 2023-06-28T20:08:53.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.369+0000 7f0f28b49700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f240b4f20 0x7f0f2415d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:53.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.369+0000 7f0f28b49700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f240d0990 con 0x7f0f240d0e20 2023-06-28T20:08:53.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.369+0000 7f0f28b49700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f240d1990 con 0x7f0f2415ceb0 2023-06-28T20:08:53.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.369+0000 7f0f28b49700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f240b8960 con 0x7f0f240b4f20 2023-06-28T20:08:53.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.369+0000 7f0f22d9d700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0f240d0e20 0x7f0f240d1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44974/0 (socket says 172.21.15.43:44974) 2023-06-28T20:08:53.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.369+0000 7f0f22d9d700 1 -- 172.21.15.43:0/195908138 learned_addr learned my addr 172.21.15.43:0/195908138 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:53.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.369+0000 7f0f21d9b700 1 --2- 172.21.15.43:0/195908138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f240b4f20 0x7f0f2415d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:53.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.369+0000 7f0f2159a700 1 -- 172.21.15.43:0/195908138 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1962664227 0 0) 0x7f0f240d0990 con 0x7f0f240d0e20 2023-06-28T20:08:53.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.369+0000 7f0f2159a700 1 -- 172.21.15.43:0/195908138 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f100035b0 con 0x7f0f240d0e20 2023-06-28T20:08:53.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f2159a700 1 -- 172.21.15.43:0/195908138 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3718723898 0 0) 0x7f0f100035b0 con 0x7f0f240d0e20 2023-06-28T20:08:53.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f2159a700 1 -- 172.21.15.43:0/195908138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f240b4f20 msgr2=0x7f0f2415d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:53.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f2159a700 1 --2- 172.21.15.43:0/195908138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f240b4f20 0x7f0f2415d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:53.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f2159a700 1 -- 172.21.15.43:0/195908138 >> v1:172.21.15.178:6789/0 conn(0x7f0f2415ceb0 legacy=0x7f0f240cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:53.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f2159a700 1 -- 172.21.15.43:0/195908138 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2415d990 con 0x7f0f240d0e20 2023-06-28T20:08:53.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f2159a700 1 -- 172.21.15.43:0/195908138 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f0f1c0019f0 con 0x7f0f240d0e20 2023-06-28T20:08:53.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f2159a700 1 -- 172.21.15.43:0/195908138 >> v1:172.21.15.43:6789/0 conn(0x7f0f240d0e20 legacy=0x7f0f240d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:53.377 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f2159a700 1 --2- 172.21.15.43:0/195908138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f10003a30 0x7f0f10003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:53.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f2159a700 1 --2- 172.21.15.43:0/195908138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f240b4f20 0x7f0f10004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:53.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f2159a700 1 --2- 172.21.15.43:0/195908138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f10004d90 0x7f0f10005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:53.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f2159a700 1 -- 172.21.15.43:0/195908138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f240b8960 con 0x7f0f10003a30 2023-06-28T20:08:53.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f2159a700 1 -- 172.21.15.43:0/195908138 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1000bac0 con 0x7f0f10004d90 2023-06-28T20:08:53.378 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f2159a700 1 -- 172.21.15.43:0/195908138 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1000bc50 con 0x7f0f240b4f20 2023-06-28T20:08:53.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f2259c700 1 --2- 172.21.15.43:0/195908138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f10003a30 0x7f0f10003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:53.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f21d9b700 1 --2- 172.21.15.43:0/195908138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f240b4f20 0x7f0f10004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:53.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f22d9d700 1 --2- 172.21.15.43:0/195908138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f10004d90 0x7f0f10005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:53.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f2259c700 1 -- 172.21.15.43:0/195908138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f240b4f20 msgr2=0x7f0f10004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:53.379 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f2259c700 1 --2- 172.21.15.43:0/195908138 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f240b4f20 0x7f0f10004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:53.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f2259c700 1 -- 172.21.15.43:0/195908138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f10004d90 msgr2=0x7f0f10005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:53.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f2259c700 1 --2- 172.21.15.43:0/195908138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f10004d90 0x7f0f10005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:53.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f2259c700 1 -- 172.21.15.43:0/195908138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f100035b0 con 0x7f0f10003a30 2023-06-28T20:08:53.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f22d9d700 1 --2- 172.21.15.43:0/195908138 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f10004d90 0x7f0f10005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:53.380 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f2259c700 1 --2- 172.21.15.43:0/195908138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f10003a30 0x7f0f10003e00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0f140018d0 tx=0x7f0f10005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7ae6c6b3bc6a server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:53.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f2159a700 1 -- 172.21.15.43:0/195908138 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1400c690 con 0x7f0f10003a30 2023-06-28T20:08:53.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f2159a700 1 -- 172.21.15.43:0/195908138 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1400c810 con 0x7f0f10003a30 2023-06-28T20:08:53.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.370+0000 7f0f2159a700 1 -- 172.21.15.43:0/195908138 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1400cbc0 con 0x7f0f10003a30 2023-06-28T20:08:53.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.371+0000 7f0f28b49700 1 -- 172.21.15.43:0/195908138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f10003a30 msgr2=0x7f0f10003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:53.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.371+0000 7f0f28b49700 1 --2- 172.21.15.43:0/195908138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f10003a30 0x7f0f10003e00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f0f140018d0 tx=0x7f0f10005880 comp rx=0 tx=0).stop 2023-06-28T20:08:53.381 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.371+0000 7f0f22d9d700 1 -- 172.21.15.43:0/195908138 reap_dead start 2023-06-28T20:08:53.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.371+0000 7f0f28b49700 1 -- 172.21.15.43:0/195908138 shutdown_connections 2023-06-28T20:08:53.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.371+0000 7f0f28b49700 1 -- 172.21.15.43:0/195908138 >> 172.21.15.43:0/195908138 conn(0x7f0f2406a5c0 msgr2=0x7f0f2406a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:53.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.371+0000 7f0f28b49700 1 -- 172.21.15.43:0/195908138 shutdown_connections 2023-06-28T20:08:53.382 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.371+0000 7f0f28b49700 1 -- 172.21.15.43:0/195908138 wait complete. 2023-06-28T20:08:53.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.372+0000 7f0f28b49700 1 Processor -- start 2023-06-28T20:08:53.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.372+0000 7f0f28b49700 1 -- start start 2023-06-28T20:08:53.385 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.373+0000 7f0f28b49700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f10003a30 0x7f0f240b79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:53.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.373+0000 7f0f28b49700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f24149240 0x7f0f24156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:53.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.373+0000 7f0f28b49700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f240b6920 0x7f0f240b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:53.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.373+0000 7f0f28b49700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f240b7300 con 0x7f0f24149240 2023-06-28T20:08:53.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.373+0000 7f0f28b49700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f24159990 con 0x7f0f10003a30 2023-06-28T20:08:53.386 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.373+0000 7f0f28b49700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f240b7f10 con 0x7f0f240b6920 2023-06-28T20:08:53.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.373+0000 7f0f21d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f24149240 0x7f0f24156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:53.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.373+0000 7f0f21d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f24149240 0x7f0f24156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55798/0 (socket says 172.21.15.43:55798) 2023-06-28T20:08:53.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.373+0000 7f0f21d9b700 1 -- 172.21.15.43:0/4152527143 learned_addr learned my addr 172.21.15.43:0/4152527143 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:53.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.374+0000 7f0f2259c700 1 --2- 172.21.15.43:0/4152527143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f10003a30 0x7f0f240b79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:53.387 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.374+0000 7f0f22d9d700 1 --2- 172.21.15.43:0/4152527143 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f240b6920 0x7f0f240b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:53.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.374+0000 7f0f21d9b700 1 -- 172.21.15.43:0/4152527143 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f240b6920 msgr2=0x7f0f240b6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:53.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.374+0000 7f0f21d9b700 1 --2- 172.21.15.43:0/4152527143 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f0f240b6920 0x7f0f240b6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:53.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.374+0000 7f0f21d9b700 1 -- 172.21.15.43:0/4152527143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f10003a30 msgr2=0x7f0f240b79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:53.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.374+0000 7f0f21d9b700 1 --2- 172.21.15.43:0/4152527143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f10003a30 0x7f0f240b79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:53.388 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.374+0000 7f0f21d9b700 1 -- 172.21.15.43:0/4152527143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f24158d80 con 0x7f0f24149240 2023-06-28T20:08:53.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.374+0000 7f0f2259c700 1 --2- 172.21.15.43:0/4152527143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f10003a30 0x7f0f240b79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:53.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.374+0000 7f0f21d9b700 1 --2- 172.21.15.43:0/4152527143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f24149240 0x7f0f24156f30 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c0119f0 tx=0x7f0f0c011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:53.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.374+0000 7f0f1affd700 1 -- 172.21.15.43:0/4152527143 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c00fa00 con 0x7f0f24149240 2023-06-28T20:08:53.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.374+0000 7f0f1affd700 1 -- 172.21.15.43:0/4152527143 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0c00fb60 con 0x7f0f24149240 2023-06-28T20:08:53.389 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.374+0000 7f0f28b49700 1 -- 172.21.15.43:0/4152527143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f24159060 con 0x7f0f24149240 2023-06-28T20:08:53.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.374+0000 7f0f1affd700 1 -- 172.21.15.43:0/4152527143 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c014440 con 0x7f0f24149240 2023-06-28T20:08:53.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.374+0000 7f0f28b49700 1 -- 172.21.15.43:0/4152527143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f24159620 con 0x7f0f24149240 2023-06-28T20:08:53.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.375+0000 7f0f28b49700 1 -- 172.21.15.43:0/4152527143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0f240cd900 con 0x7f0f24149240 2023-06-28T20:08:53.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.375+0000 7f0f1affd700 1 -- 172.21.15.43:0/4152527143 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0f0c01d400 con 0x7f0f24149240 2023-06-28T20:08:53.390 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.376+0000 7f0f1affd700 1 --2- 172.21.15.43:0/4152527143 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0f0406ace0 0x7f0f0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:53.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.376+0000 7f0f1affd700 1 -- 172.21.15.43:0/4152527143 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f0f0c089cc0 con 0x7f0f24149240 2023-06-28T20:08:53.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.376+0000 7f0f2259c700 1 --2- 172.21.15.43:0/4152527143 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0f0406ace0 0x7f0f0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:53.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.376+0000 7f0f1affd700 1 --2- 172.21.15.43:0/4152527143 >> [v2:172.21.15.43:6816/2901255345,v1:172.21.15.43:6817/2901255345] conn(0x7f0f04071150 0x7f0f04073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:53.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.376+0000 7f0f1affd700 1 -- 172.21.15.43:0/4152527143 --> [v2:172.21.15.43:6816/2901255345,v1:172.21.15.43:6817/2901255345] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f04073c30 con 0x7f0f04071150 2023-06-28T20:08:53.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.376+0000 7f0f1affd700 1 -- 172.21.15.43:0/4152527143 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f0c08a0f0 con 0x7f0f24149240 2023-06-28T20:08:53.391 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.376+0000 7f0f22d9d700 1 --2- 172.21.15.43:0/4152527143 >> [v2:172.21.15.43:6816/2901255345,v1:172.21.15.43:6817/2901255345] conn(0x7f0f04071150 0x7f0f04073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:53.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.376+0000 7f0f2259c700 1 --2- 172.21.15.43:0/4152527143 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0f0406ace0 0x7f0f0406d190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0f14014040 tx=0x7f0f14005040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:53.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.377+0000 7f0f22d9d700 1 --2- 172.21.15.43:0/4152527143 >> [v2:172.21.15.43:6816/2901255345,v1:172.21.15.43:6817/2901255345] conn(0x7f0f04071150 0x7f0f04073560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:53.392 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.377+0000 7f0f1affd700 1 -- 172.21.15.43:0/4152527143 <== osd.3 v2:172.21.15.43:6816/2901255345 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0f04073c30 con 0x7f0f04071150 2023-06-28T20:08:53.394 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.392+0000 7f0f28b49700 1 -- 172.21.15.43:0/4152527143 --> [v2:172.21.15.43:6816/2901255345,v1:172.21.15.43:6817/2901255345] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0f240cd620 con 0x7f0f04071150 2023-06-28T20:08:53.395 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.392+0000 7f0f1affd700 1 -- 172.21.15.43:0/4152527143 <== osd.3 v2:172.21.15.43:6816/2901255345 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f0f1c001930 con 0x7f0f04071150 2023-06-28T20:08:53.395 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-06-28T20:08:53.395 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-28T20:08:53.396 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-06-28T20:08:53.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.392+0000 7f0f28b49700 1 -- 172.21.15.43:0/4152527143 >> [v2:172.21.15.43:6816/2901255345,v1:172.21.15.43:6817/2901255345] conn(0x7f0f04071150 msgr2=0x7f0f04073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:53.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.392+0000 7f0f28b49700 1 --2- 172.21.15.43:0/4152527143 >> [v2:172.21.15.43:6816/2901255345,v1:172.21.15.43:6817/2901255345] conn(0x7f0f04071150 0x7f0f04073560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:53.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.392+0000 7f0f28b49700 1 -- 172.21.15.43:0/4152527143 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0f0406ace0 msgr2=0x7f0f0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:53.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.392+0000 7f0f28b49700 1 --2- 172.21.15.43:0/4152527143 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f0f0406ace0 0x7f0f0406d190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0f14014040 tx=0x7f0f14005040 comp rx=0 tx=0).stop 2023-06-28T20:08:53.397 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.392+0000 7f0f28b49700 1 -- 172.21.15.43:0/4152527143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f24149240 msgr2=0x7f0f24156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:53.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.392+0000 7f0f28b49700 1 --2- 172.21.15.43:0/4152527143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f24149240 0x7f0f24156f30 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c0119f0 tx=0x7f0f0c011db0 comp rx=0 tx=0).stop 2023-06-28T20:08:53.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.392+0000 7f0f22d9d700 1 -- 172.21.15.43:0/4152527143 reap_dead start 2023-06-28T20:08:53.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.392+0000 7f0f28b49700 1 -- 172.21.15.43:0/4152527143 shutdown_connections 2023-06-28T20:08:53.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.392+0000 7f0f28b49700 1 -- 172.21.15.43:0/4152527143 >> 172.21.15.43:0/4152527143 conn(0x7f0f2406a5c0 msgr2=0x7f0f2415dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:53.398 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.392+0000 7f0f28b49700 1 -- 172.21.15.43:0/4152527143 shutdown_connections 2023-06-28T20:08:53.399 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.392+0000 7f0f28b49700 1 -- 172.21.15.43:0/4152527143 wait complete. 2023-06-28T20:08:53.407 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-06-28T20:08:53.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.529+0000 7fe94adfd700 1 Processor -- start 2023-06-28T20:08:53.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.529+0000 7fe94adfd700 1 -- start start 2023-06-28T20:08:53.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.529+0000 7fe94adfd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe9440cb3c0 0x7fe94406c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:53.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.529+0000 7fe94adfd700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fe94406c5e0 0x7fe944071060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:53.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.529+0000 7fe94adfd700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe94406b940 con 0x7fe9440caff0 2023-06-28T20:08:53.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.529+0000 7fe94adfd700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe94406ca60 con 0x7fe9440cb3c0 2023-06-28T20:08:53.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.529+0000 7fe94adfd700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe94406a020 con 0x7fe94406c5e0 2023-06-28T20:08:53.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.530+0000 7fe943fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe9440cb3c0 0x7fe94406c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:53.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.530+0000 7fe943fff700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe9440cb3c0 0x7fe94406c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55806/0 (socket says 172.21.15.43:55806) 2023-06-28T20:08:53.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.530+0000 7fe943fff700 1 -- 172.21.15.43:0/3169743561 learned_addr learned my addr 172.21.15.43:0/3169743561 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:53.535 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.530+0000 7fe94939a700 1 --2- 172.21.15.43:0/3169743561 >> v2:172.21.15.178:3300/0 conn(0x7fe94406c5e0 0x7fe944071060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:53.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.530+0000 7fe943fff700 1 -- 172.21.15.43:0/3169743561 >> v2:172.21.15.178:3300/0 conn(0x7fe94406c5e0 msgr2=0x7fe944071060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:53.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.530+0000 7fe943fff700 1 --2- 172.21.15.43:0/3169743561 >> v2:172.21.15.178:3300/0 conn(0x7fe94406c5e0 0x7fe944071060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:53.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.530+0000 7fe943fff700 1 -- 172.21.15.43:0/3169743561 >> v1:172.21.15.178:6789/0 conn(0x7fe9440caff0 legacy=0x7fe94406b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:53.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.530+0000 7fe943fff700 1 -- 172.21.15.43:0/3169743561 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94406a300 con 0x7fe9440cb3c0 2023-06-28T20:08:53.536 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.530+0000 7fe943fff700 1 --2- 172.21.15.43:0/3169743561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9440cb3c0 0x7fe94406c0a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe9300098f0 tx=0x7fe930009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faac8463bda1442c server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:53.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.530+0000 7fe9437fe700 1 -- 172.21.15.43:0/3169743561 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe930004230 con 0x7fe9440cb3c0 2023-06-28T20:08:53.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.530+0000 7fe9437fe700 1 -- 172.21.15.43:0/3169743561 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe930004390 con 0x7fe9440cb3c0 2023-06-28T20:08:53.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.530+0000 7fe9437fe700 1 -- 172.21.15.43:0/3169743561 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9300036a0 con 0x7fe9440cb3c0 2023-06-28T20:08:53.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.531+0000 7fe94adfd700 1 -- 172.21.15.43:0/3169743561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9440cb3c0 msgr2=0x7fe94406c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:53.537 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.531+0000 7fe94adfd700 1 --2- 172.21.15.43:0/3169743561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9440cb3c0 0x7fe94406c0a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe9300098f0 tx=0x7fe930009c00 comp rx=0 tx=0).stop 2023-06-28T20:08:53.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.531+0000 7fe94adfd700 1 -- 172.21.15.43:0/3169743561 shutdown_connections 2023-06-28T20:08:53.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.531+0000 7fe94adfd700 1 --2- 172.21.15.43:0/3169743561 >> v2:172.21.15.178:3300/0 conn(0x7fe94406c5e0 0x7fe944071060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:53.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.531+0000 7fe94adfd700 1 --2- 172.21.15.43:0/3169743561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9440cb3c0 0x7fe94406c0a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:53.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.531+0000 7fe94adfd700 1 -- 172.21.15.43:0/3169743561 >> 172.21.15.43:0/3169743561 conn(0x7fe9440b1a50 msgr2=0x7fe9440cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:53.538 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.531+0000 7fe94adfd700 1 -- 172.21.15.43:0/3169743561 shutdown_connections 2023-06-28T20:08:53.539 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.531+0000 7fe94adfd700 1 -- 172.21.15.43:0/3169743561 wait complete. 2023-06-28T20:08:53.540 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.532+0000 7fe94adfd700 1 Processor -- start 2023-06-28T20:08:53.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.532+0000 7fe94adfd700 1 -- start start 2023-06-28T20:08:53.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.533+0000 7fe94adfd700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe94406c5e0 0x7fe9440b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:53.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.533+0000 7fe94adfd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9440caff0 0x7fe9441e2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:53.541 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.533+0000 7fe94adfd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9440cb3c0 0x7fe944156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.533+0000 7fe94adfd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9440b39b0 con 0x7fe9440caff0 2023-06-28T20:08:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.533+0000 7fe94adfd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9440b2930 con 0x7fe9440cb3c0 2023-06-28T20:08:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.533+0000 7fe94adfd700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9440b2c10 con 0x7fe94406c5e0 2023-06-28T20:08:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.533+0000 7fe943fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9440caff0 0x7fe9441e2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:53.542 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.533+0000 7fe943fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9440caff0 0x7fe9441e2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55812/0 (socket says 172.21.15.43:55812) 2023-06-28T20:08:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.533+0000 7fe943fff700 1 -- 172.21.15.43:0/373289179 learned_addr learned my addr 172.21.15.43:0/373289179 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.533+0000 7fe94939a700 1 --2- 172.21.15.43:0/373289179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9440cb3c0 0x7fe944156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.533+0000 7fe948b99700 1 --2- 172.21.15.43:0/373289179 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe94406c5e0 0x7fe9440b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.533+0000 7fe943fff700 1 -- 172.21.15.43:0/373289179 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe94406c5e0 msgr2=0x7fe9440b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:53.543 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.533+0000 7fe943fff700 1 --2- 172.21.15.43:0/373289179 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe94406c5e0 0x7fe9440b20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:53.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.533+0000 7fe943fff700 1 -- 172.21.15.43:0/373289179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9440cb3c0 msgr2=0x7fe944156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:53.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.533+0000 7fe943fff700 1 --2- 172.21.15.43:0/373289179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9440cb3c0 0x7fe944156b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:53.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.533+0000 7fe943fff700 1 -- 172.21.15.43:0/373289179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe930009550 con 0x7fe9440caff0 2023-06-28T20:08:53.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.534+0000 7fe943fff700 1 --2- 172.21.15.43:0/373289179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9440caff0 0x7fe9441e2590 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe930009130 tx=0x7fe930004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:53.544 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.534+0000 7fe9417fa700 1 -- 172.21.15.43:0/373289179 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe930003b50 con 0x7fe9440caff0 2023-06-28T20:08:53.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.534+0000 7fe9417fa700 1 -- 172.21.15.43:0/373289179 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93001f070 con 0x7fe9440caff0 2023-06-28T20:08:53.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.534+0000 7fe9417fa700 1 -- 172.21.15.43:0/373289179 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe930011630 con 0x7fe9440caff0 2023-06-28T20:08:53.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.534+0000 7fe94adfd700 1 -- 172.21.15.43:0/373289179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9440b2ef0 con 0x7fe9440caff0 2023-06-28T20:08:53.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.534+0000 7fe94adfd700 1 -- 172.21.15.43:0/373289179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9440b34b0 con 0x7fe9440caff0 2023-06-28T20:08:53.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.535+0000 7fe94adfd700 1 -- 172.21.15.43:0/373289179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9440b3640 con 0x7fe9440caff0 2023-06-28T20:08:53.545 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.535+0000 7fe9417fa700 1 -- 172.21.15.43:0/373289179 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe930021030 con 0x7fe9440caff0 2023-06-28T20:08:53.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.536+0000 7fe9417fa700 1 --2- 172.21.15.43:0/373289179 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe92806ace0 0x7fe92806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:53.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.536+0000 7fe9417fa700 1 -- 172.21.15.43:0/373289179 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe93008a510 con 0x7fe9440caff0 2023-06-28T20:08:53.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.536+0000 7fe948b99700 1 --2- 172.21.15.43:0/373289179 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe92806ace0 0x7fe92806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:53.546 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.536+0000 7fe948b99700 1 --2- 172.21.15.43:0/373289179 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe92806ace0 0x7fe92806d190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe9440b42c0 tx=0x7fe938007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:53.547 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.540+0000 7fe9417fa700 1 -- 172.21.15.43:0/373289179 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe9300558d0 con 0x7fe9440caff0 2023-06-28T20:08:53.741 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.738+0000 7fe94adfd700 1 -- 172.21.15.43:0/373289179 --> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe94406a8c0 con 0x7fe92806ace0 2023-06-28T20:08:54.141 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.738+0000 7fe9417fa700 1 -- 172.21.15.43:0/373289179 <== mgr.4101 v2:172.21.15.43:6832/406144153 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe938003cf0 con 0x7fe92806ace0 2023-06-28T20:08:54.144 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.740+0000 7fe94adfd700 1 -- 172.21.15.43:0/373289179 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe92806ace0 msgr2=0x7fe92806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.740+0000 7fe94adfd700 1 --2- 172.21.15.43:0/373289179 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe92806ace0 0x7fe92806d190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe9440b42c0 tx=0x7fe938007ec0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.740+0000 7fe94adfd700 1 -- 172.21.15.43:0/373289179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9440caff0 msgr2=0x7fe9441e2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.145 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.740+0000 7fe94adfd700 1 --2- 172.21.15.43:0/373289179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9440caff0 0x7fe9441e2590 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe930009130 tx=0x7fe930004750 comp rx=0 tx=0).stop 2023-06-28T20:08:54.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.740+0000 7fe94adfd700 1 -- 172.21.15.43:0/373289179 shutdown_connections 2023-06-28T20:08:54.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.740+0000 7fe94adfd700 1 --2- 172.21.15.43:0/373289179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe9440cb3c0 0x7fe944156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.740+0000 7fe94adfd700 1 --2- 172.21.15.43:0/373289179 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fe92806ace0 0x7fe92806d190 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.740+0000 7fe94adfd700 1 --2- 172.21.15.43:0/373289179 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fe94406c5e0 0x7fe9440b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.146 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.740+0000 7fe94adfd700 1 --2- 172.21.15.43:0/373289179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9440caff0 0x7fe9441e2590 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.740+0000 7fe94adfd700 1 -- 172.21.15.43:0/373289179 >> 172.21.15.43:0/373289179 conn(0x7fe9440b1a50 msgr2=0x7fe9440c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:54.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.740+0000 7fe94adfd700 1 -- 172.21.15.43:0/373289179 shutdown_connections 2023-06-28T20:08:54.147 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:53.740+0000 7fe94adfd700 1 -- 172.21.15.43:0/373289179 wait complete. 2023-06-28T20:08:54.147 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 3 to deep-scrub 2023-06-28T20:08:54.148 INFO:tasks.ceph:Scrubbing osd.4 2023-06-28T20:08:54.148 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-06-28T20:08:54.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.279+0000 7f81ff74f700 1 Processor -- start 2023-06-28T20:08:54.283 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.279+0000 7f81ff74f700 1 -- start start 2023-06-28T20:08:54.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.280+0000 7f81ff74f700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f81f80cde50 0x7f81f806e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:54.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.280+0000 7f81ff74f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f806d1b0 con 0x7f81f8070900 2023-06-28T20:08:54.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.280+0000 7f81ff74f700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f806d490 con 0x7f81f815a670 2023-06-28T20:08:54.285 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.280+0000 7f81ff74f700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f806e790 con 0x7f81f80cde50 2023-06-28T20:08:54.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.280+0000 7f81fccea700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f81f8070900 0x7f81f80cd740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45004/0 (socket says 172.21.15.43:45004) 2023-06-28T20:08:54.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.280+0000 7f81fccea700 1 -- 172.21.15.43:0/3663133674 learned_addr learned my addr 172.21.15.43:0/3663133674 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:54.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.280+0000 7f81fdcec700 1 --2- 172.21.15.43:0/3663133674 >> v2:172.21.15.178:3300/0 conn(0x7f81f80cde50 0x7f81f806e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:54.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.281+0000 7f81effff700 1 -- 172.21.15.43:0/3663133674 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2223782696 0 0) 0x7f81f806d1b0 con 0x7f81f8070900 2023-06-28T20:08:54.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.281+0000 7f81effff700 1 -- 172.21.15.43:0/3663133674 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81e40057c0 con 0x7f81f8070900 2023-06-28T20:08:54.286 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.281+0000 7f81fdcec700 1 -- 172.21.15.43:0/3663133674 >> v1:172.21.15.178:6789/0 conn(0x7f81f815a670 legacy=0x7f81f806c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.281+0000 7f81effff700 1 -- 172.21.15.43:0/3663133674 <== mon.1 v1:172.21.15.178:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3968042805 0 0) 0x7f81f806d490 con 0x7f81f815a670 2023-06-28T20:08:54.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.281+0000 7f81fdcec700 1 -- 172.21.15.43:0/3663133674 >> v1:172.21.15.43:6789/0 conn(0x7f81f8070900 legacy=0x7f81f80cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.281+0000 7f81fdcec700 1 -- 172.21.15.43:0/3663133674 --> v2:172.21.15.178:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f806d770 con 0x7f81f80cde50 2023-06-28T20:08:54.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.281+0000 7f81fdcec700 1 --2- 172.21.15.43:0/3663133674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81f80cde50 0x7f81f806e250 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009d90 tx=0x7f81f000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46c8b6e12b9b4f3e server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:54.287 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.281+0000 7f81effff700 1 -- 172.21.15.43:0/3663133674 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f000e070 con 0x7f81f80cde50 2023-06-28T20:08:54.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.281+0000 7f81effff700 1 -- 172.21.15.43:0/3663133674 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f0003a20 con 0x7f81f80cde50 2023-06-28T20:08:54.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.281+0000 7f81effff700 1 -- 172.21.15.43:0/3663133674 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f40015f0 con 0x7f81f80cde50 2023-06-28T20:08:54.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.282+0000 7f81ff74f700 1 -- 172.21.15.43:0/3663133674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81f80cde50 msgr2=0x7f81f806e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.282+0000 7f81ff74f700 1 --2- 172.21.15.43:0/3663133674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81f80cde50 0x7f81f806e250 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f81f0009d90 tx=0x7f81f000b330 comp rx=0 tx=0).stop 2023-06-28T20:08:54.288 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.282+0000 7f81ff74f700 1 -- 172.21.15.43:0/3663133674 shutdown_connections 2023-06-28T20:08:54.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.282+0000 7f81ff74f700 1 --2- 172.21.15.43:0/3663133674 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81f80cde50 0x7f81f806e250 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.289 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.282+0000 7f81ff74f700 1 -- 172.21.15.43:0/3663133674 >> 172.21.15.43:0/3663133674 conn(0x7f81f80cd240 msgr2=0x7f81f80cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:54.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.282+0000 7f81ff74f700 1 -- 172.21.15.43:0/3663133674 shutdown_connections 2023-06-28T20:08:54.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.282+0000 7f81ff74f700 1 -- 172.21.15.43:0/3663133674 wait complete. 2023-06-28T20:08:54.292 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.283+0000 7f81ff74f700 1 Processor -- start 2023-06-28T20:08:54.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.283+0000 7f81ff74f700 1 -- start start 2023-06-28T20:08:54.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.284+0000 7f81ff74f700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81f8070900 0x7f81f80b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:54.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.284+0000 7f81ff74f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81f80cde50 0x7f81f80c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:54.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.284+0000 7f81ff74f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81f815a670 0x7f81f8146dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:54.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.284+0000 7f81ff74f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f80c3830 con 0x7f81f80cde50 2023-06-28T20:08:54.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.284+0000 7f81ff74f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f815c3a0 con 0x7f81f815a670 2023-06-28T20:08:54.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.284+0000 7f81ff74f700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f815c680 con 0x7f81f8070900 2023-06-28T20:08:54.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.284+0000 7f81fccea700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81f80cde50 0x7f81f80c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:54.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.284+0000 7f81fccea700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81f80cde50 0x7f81f80c60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55824/0 (socket says 172.21.15.43:55824) 2023-06-28T20:08:54.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.284+0000 7f81fccea700 1 -- 172.21.15.43:0/3547999429 learned_addr learned my addr 172.21.15.43:0/3547999429 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:54.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.284+0000 7f81fdcec700 1 --2- 172.21.15.43:0/3547999429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81f815a670 0x7f81f8146dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:54.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.284+0000 7f81fd4eb700 1 --2- 172.21.15.43:0/3547999429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81f8070900 0x7f81f80b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:54.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.285+0000 7f81fccea700 1 -- 172.21.15.43:0/3547999429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81f8070900 msgr2=0x7f81f80b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.285+0000 7f81fccea700 1 --2- 172.21.15.43:0/3547999429 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f81f8070900 0x7f81f80b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.285+0000 7f81fccea700 1 -- 172.21.15.43:0/3547999429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81f815a670 msgr2=0x7f81f8146dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.285+0000 7f81fccea700 1 --2- 172.21.15.43:0/3547999429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81f815a670 0x7f81f8146dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.285+0000 7f81fccea700 1 -- 172.21.15.43:0/3547999429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f0009a70 con 0x7f81f80cde50 2023-06-28T20:08:54.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.285+0000 7f81fccea700 1 --2- 172.21.15.43:0/3547999429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81f80cde50 0x7f81f80c60c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f81f40011e0 tx=0x7f81f400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:54.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.285+0000 7f81edffb700 1 -- 172.21.15.43:0/3547999429 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f4004c80 con 0x7f81f80cde50 2023-06-28T20:08:54.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.285+0000 7f81edffb700 1 -- 172.21.15.43:0/3547999429 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f4004de0 con 0x7f81f80cde50 2023-06-28T20:08:54.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.285+0000 7f81ff74f700 1 -- 172.21.15.43:0/3547999429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f815c960 con 0x7f81f80cde50 2023-06-28T20:08:54.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.285+0000 7f81edffb700 1 -- 172.21.15.43:0/3547999429 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f40072f0 con 0x7f81f80cde50 2023-06-28T20:08:54.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.285+0000 7f81ff74f700 1 -- 172.21.15.43:0/3547999429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f80b4420 con 0x7f81f80cde50 2023-06-28T20:08:54.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.286+0000 7f81ff74f700 1 -- 172.21.15.43:0/3547999429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f81f806dd30 con 0x7f81f80cde50 2023-06-28T20:08:54.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.286+0000 7f81edffb700 1 -- 172.21.15.43:0/3547999429 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f81f4007450 con 0x7f81f80cde50 2023-06-28T20:08:54.297 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.287+0000 7f81edffb700 1 --2- 172.21.15.43:0/3547999429 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f81dc06ad10 0x7f81dc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:54.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.287+0000 7f81edffb700 1 -- 172.21.15.43:0/3547999429 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f81f4088f90 con 0x7f81f80cde50 2023-06-28T20:08:54.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.287+0000 7f81fd4eb700 1 --2- 172.21.15.43:0/3547999429 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f81dc06ad10 0x7f81dc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:54.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.287+0000 7f81edffb700 1 --2- 172.21.15.43:0/3547999429 >> [v2:172.21.15.178:6820/2918661202,v1:172.21.15.178:6822/2918661202] conn(0x7f81dc071180 0x7f81dc073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:54.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.287+0000 7f81edffb700 1 -- 172.21.15.43:0/3547999429 --> [v2:172.21.15.178:6820/2918661202,v1:172.21.15.178:6822/2918661202] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f81dc073c60 con 0x7f81dc071180 2023-06-28T20:08:54.298 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.287+0000 7f81edffb700 1 -- 172.21.15.43:0/3547999429 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81f408b5d0 con 0x7f81f80cde50 2023-06-28T20:08:54.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.287+0000 7f81fdcec700 1 --2- 172.21.15.43:0/3547999429 >> [v2:172.21.15.178:6820/2918661202,v1:172.21.15.178:6822/2918661202] conn(0x7f81dc071180 0x7f81dc073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:54.299 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.287+0000 7f81fd4eb700 1 --2- 172.21.15.43:0/3547999429 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f81dc06ad10 0x7f81dc06d1c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f81e8001120 tx=0x7f81e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:54.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.288+0000 7f81fdcec700 1 --2- 172.21.15.43:0/3547999429 >> [v2:172.21.15.178:6820/2918661202,v1:172.21.15.178:6822/2918661202] conn(0x7f81dc071180 0x7f81dc073590 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:54.300 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.288+0000 7f81edffb700 1 -- 172.21.15.43:0/3547999429 <== osd.4 v2:172.21.15.178:6820/2918661202 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f81dc073c60 con 0x7f81dc071180 2023-06-28T20:08:54.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.303+0000 7f81ff74f700 1 -- 172.21.15.43:0/3547999429 --> [v2:172.21.15.178:6820/2918661202,v1:172.21.15.178:6822/2918661202] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f81f806da50 con 0x7f81dc071180 2023-06-28T20:08:54.306 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.304+0000 7f81edffb700 1 -- 172.21.15.43:0/3547999429 <== osd.4 v2:172.21.15.178:6820/2918661202 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f81f000cab0 con 0x7f81dc071180 2023-06-28T20:08:54.307 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-06-28T20:08:54.307 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-28T20:08:54.307 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-06-28T20:08:54.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.304+0000 7f81ff74f700 1 -- 172.21.15.43:0/3547999429 >> [v2:172.21.15.178:6820/2918661202,v1:172.21.15.178:6822/2918661202] conn(0x7f81dc071180 msgr2=0x7f81dc073590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.308 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.304+0000 7f81ff74f700 1 --2- 172.21.15.43:0/3547999429 >> [v2:172.21.15.178:6820/2918661202,v1:172.21.15.178:6822/2918661202] conn(0x7f81dc071180 0x7f81dc073590 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.304+0000 7f81ff74f700 1 -- 172.21.15.43:0/3547999429 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f81dc06ad10 msgr2=0x7f81dc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.304+0000 7f81ff74f700 1 --2- 172.21.15.43:0/3547999429 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f81dc06ad10 0x7f81dc06d1c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f81e8001120 tx=0x7f81e8009040 comp rx=0 tx=0).stop 2023-06-28T20:08:54.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.304+0000 7f81ff74f700 1 -- 172.21.15.43:0/3547999429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81f80cde50 msgr2=0x7f81f80c60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.309 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.304+0000 7f81ff74f700 1 --2- 172.21.15.43:0/3547999429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81f80cde50 0x7f81f80c60c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f81f40011e0 tx=0x7f81f400c9b0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.304+0000 7f81fdcec700 1 -- 172.21.15.43:0/3547999429 reap_dead start 2023-06-28T20:08:54.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.304+0000 7f81ff74f700 1 -- 172.21.15.43:0/3547999429 shutdown_connections 2023-06-28T20:08:54.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.304+0000 7f81ff74f700 1 -- 172.21.15.43:0/3547999429 >> 172.21.15.43:0/3547999429 conn(0x7f81f80cd240 msgr2=0x7f81f8157280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:54.310 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.304+0000 7f81ff74f700 1 -- 172.21.15.43:0/3547999429 shutdown_connections 2023-06-28T20:08:54.311 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.304+0000 7f81ff74f700 1 -- 172.21.15.43:0/3547999429 wait complete. 2023-06-28T20:08:54.319 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-06-28T20:08:54.488 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.485+0000 7f7d003c8700 1 Processor -- start 2023-06-28T20:08:54.489 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.485+0000 7f7d003c8700 1 -- start start 2023-06-28T20:08:54.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.486+0000 7f7d003c8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf80b1f90 0x7f7cf8070900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:54.492 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.486+0000 7f7d003c8700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cf8158290 con 0x7f7cf8157ec0 2023-06-28T20:08:54.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.486+0000 7f7d003c8700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cf815ab30 con 0x7f7cf8070e40 2023-06-28T20:08:54.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.486+0000 7f7d003c8700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf815a920 con 0x7f7cf80b1f90 2023-06-28T20:08:54.493 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.486+0000 7f7cfe164700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7cf8157ec0 0x7f7cf80b1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45018/0 (socket says 172.21.15.43:45018) 2023-06-28T20:08:54.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.486+0000 7f7cfe164700 1 -- 172.21.15.43:0/4034199043 learned_addr learned my addr 172.21.15.43:0/4034199043 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:54.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.486+0000 7f7cfd963700 1 --2- 172.21.15.43:0/4034199043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf80b1f90 0x7f7cf8070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:54.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.486+0000 7f7cfd162700 1 -- 172.21.15.43:0/4034199043 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2831252623 0 0) 0x7f7cf8158290 con 0x7f7cf8157ec0 2023-06-28T20:08:54.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.486+0000 7f7cfd162700 1 -- 172.21.15.43:0/4034199043 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ce40035b0 con 0x7f7cf8157ec0 2023-06-28T20:08:54.494 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfd162700 1 -- 172.21.15.43:0/4034199043 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1901868691 0 0) 0x7f7ce40035b0 con 0x7f7cf8157ec0 2023-06-28T20:08:54.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfd162700 1 -- 172.21.15.43:0/4034199043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf80b1f90 msgr2=0x7f7cf8070900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:54.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfd162700 1 --2- 172.21.15.43:0/4034199043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf80b1f90 0x7f7cf8070900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfd162700 1 -- 172.21.15.43:0/4034199043 >> v1:172.21.15.178:6789/0 conn(0x7f7cf8070e40 legacy=0x7f7cf815df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.495 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfd162700 1 -- 172.21.15.43:0/4034199043 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf815ae10 con 0x7f7cf8157ec0 2023-06-28T20:08:54.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfd162700 1 -- 172.21.15.43:0/4034199043 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7f7ce8001670 con 0x7f7cf8157ec0 2023-06-28T20:08:54.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfd162700 1 -- 172.21.15.43:0/4034199043 >> v1:172.21.15.43:6789/0 conn(0x7f7cf8157ec0 legacy=0x7f7cf80b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfd162700 1 --2- 172.21.15.43:0/4034199043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ce4003a30 0x7f7ce4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:54.496 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfd162700 1 --2- 172.21.15.43:0/4034199043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf80b1f90 0x7f7ce4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:54.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfd162700 1 --2- 172.21.15.43:0/4034199043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ce4004d90 0x7f7ce4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:54.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfd162700 1 -- 172.21.15.43:0/4034199043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf815a920 con 0x7f7ce4004d90 2023-06-28T20:08:54.497 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfd162700 1 -- 172.21.15.43:0/4034199043 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce400bac0 con 0x7f7ce4003a30 2023-06-28T20:08:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfd162700 1 -- 172.21.15.43:0/4034199043 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce400bc80 con 0x7f7cf80b1f90 2023-06-28T20:08:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfe965700 1 --2- 172.21.15.43:0/4034199043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ce4004d90 0x7f7ce4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfd963700 1 --2- 172.21.15.43:0/4034199043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf80b1f90 0x7f7ce4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:54.498 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfe164700 1 --2- 172.21.15.43:0/4034199043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ce4003a30 0x7f7ce4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:54.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfe965700 1 -- 172.21.15.43:0/4034199043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf80b1f90 msgr2=0x7f7ce4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:54.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfe965700 1 --2- 172.21.15.43:0/4034199043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf80b1f90 0x7f7ce4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfe965700 1 -- 172.21.15.43:0/4034199043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ce4003a30 msgr2=0x7f7ce4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:54.499 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfe965700 1 --2- 172.21.15.43:0/4034199043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ce4003a30 0x7f7ce4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfe965700 1 -- 172.21.15.43:0/4034199043 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce40035b0 con 0x7f7ce4004d90 2023-06-28T20:08:54.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfd963700 1 --2- 172.21.15.43:0/4034199043 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf80b1f90 0x7f7ce4004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:08:54.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfe164700 1 --2- 172.21.15.43:0/4034199043 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ce4003a30 0x7f7ce4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:08:54.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.487+0000 7f7cfe965700 1 --2- 172.21.15.43:0/4034199043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ce4004d90 0x7f7ce4005200 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000c240 tx=0x7f7ce4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4332bb0be2623f59 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:54.500 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.488+0000 7f7cfd162700 1 -- 172.21.15.43:0/4034199043 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0001e70 con 0x7f7ce4004d90 2023-06-28T20:08:54.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.488+0000 7f7cfd162700 1 -- 172.21.15.43:0/4034199043 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf000cde0 con 0x7f7ce4004d90 2023-06-28T20:08:54.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.488+0000 7f7cfd162700 1 -- 172.21.15.43:0/4034199043 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0012660 con 0x7f7ce4004d90 2023-06-28T20:08:54.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.488+0000 7f7d003c8700 1 -- 172.21.15.43:0/4034199043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ce4004d90 msgr2=0x7f7ce4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.501 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.488+0000 7f7d003c8700 1 --2- 172.21.15.43:0/4034199043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ce4004d90 0x7f7ce4005200 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000c240 tx=0x7f7ce4005880 comp rx=0 tx=0).stop 2023-06-28T20:08:54.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.488+0000 7f7cfe965700 1 -- 172.21.15.43:0/4034199043 reap_dead start 2023-06-28T20:08:54.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.488+0000 7f7d003c8700 1 -- 172.21.15.43:0/4034199043 shutdown_connections 2023-06-28T20:08:54.502 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.488+0000 7f7d003c8700 1 -- 172.21.15.43:0/4034199043 >> 172.21.15.43:0/4034199043 conn(0x7f7cf806b000 msgr2=0x7f7cf806b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.488+0000 7f7d003c8700 1 -- 172.21.15.43:0/4034199043 shutdown_connections 2023-06-28T20:08:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.488+0000 7f7d003c8700 1 -- 172.21.15.43:0/4034199043 wait complete. 2023-06-28T20:08:54.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.489+0000 7f7d003c8700 1 Processor -- start 2023-06-28T20:08:54.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.490+0000 7f7d003c8700 1 -- start start 2023-06-28T20:08:54.506 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.490+0000 7f7d003c8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ce4004d90 0x7f7cf814b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:54.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.490+0000 7f7d003c8700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf81554b0 0x7f7cf8156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:54.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.490+0000 7f7d003c8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf8157120 0x7f7cf81ef130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:54.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.490+0000 7f7d003c8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf815b9e0 con 0x7f7cf8157120 2023-06-28T20:08:54.507 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.490+0000 7f7d003c8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf81559b0 con 0x7f7ce4004d90 2023-06-28T20:08:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.490+0000 7f7d003c8700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf8155c90 con 0x7f7cf81554b0 2023-06-28T20:08:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.490+0000 7f7cfe965700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf8157120 0x7f7cf81ef130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.490+0000 7f7cfe965700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf8157120 0x7f7cf81ef130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55848/0 (socket says 172.21.15.43:55848) 2023-06-28T20:08:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.490+0000 7f7cfe965700 1 -- 172.21.15.43:0/2966684024 learned_addr learned my addr 172.21.15.43:0/2966684024 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.491+0000 7f7cfe164700 1 --2- 172.21.15.43:0/2966684024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ce4004d90 0x7f7cf814b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.491+0000 7f7cfd963700 1 --2- 172.21.15.43:0/2966684024 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf81554b0 0x7f7cf8156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.491+0000 7f7cfe965700 1 -- 172.21.15.43:0/2966684024 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf81554b0 msgr2=0x7f7cf8156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.491+0000 7f7cfe965700 1 --2- 172.21.15.43:0/2966684024 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf81554b0 0x7f7cf8156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.491+0000 7f7cfe965700 1 -- 172.21.15.43:0/2966684024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ce4004d90 msgr2=0x7f7cf814b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.491+0000 7f7cfe965700 1 --2- 172.21.15.43:0/2966684024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ce4004d90 0x7f7cf814b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.491+0000 7f7cfe965700 1 -- 172.21.15.43:0/2966684024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf8155f70 con 0x7f7cf8157120 2023-06-28T20:08:54.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.491+0000 7f7cfe164700 1 --2- 172.21.15.43:0/2966684024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ce4004d90 0x7f7cf814b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:54.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.491+0000 7f7cfd963700 1 --2- 172.21.15.43:0/2966684024 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf81554b0 0x7f7cf8156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:54.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.491+0000 7f7cfe965700 1 --2- 172.21.15.43:0/2966684024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf8157120 0x7f7cf81ef130 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0001520 tx=0x7f7cf0001490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:54.511 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.491+0000 7f7ceeffd700 1 -- 172.21.15.43:0/2966684024 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0003e90 con 0x7f7cf8157120 2023-06-28T20:08:54.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.491+0000 7f7ceeffd700 1 -- 172.21.15.43:0/2966684024 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf00052d0 con 0x7f7cf8157120 2023-06-28T20:08:54.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.491+0000 7f7d003c8700 1 -- 172.21.15.43:0/2966684024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf8156250 con 0x7f7cf8157120 2023-06-28T20:08:54.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.491+0000 7f7ceeffd700 1 -- 172.21.15.43:0/2966684024 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0005680 con 0x7f7cf8157120 2023-06-28T20:08:54.512 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.491+0000 7f7d003c8700 1 -- 172.21.15.43:0/2966684024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf814bd20 con 0x7f7cf8157120 2023-06-28T20:08:54.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.492+0000 7f7d003c8700 1 -- 172.21.15.43:0/2966684024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf80728b0 con 0x7f7cf8157120 2023-06-28T20:08:54.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.492+0000 7f7ceeffd700 1 -- 172.21.15.43:0/2966684024 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7cf0023070 con 0x7f7cf8157120 2023-06-28T20:08:54.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.493+0000 7f7ceeffd700 1 --2- 172.21.15.43:0/2966684024 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7cdc06ace0 0x7f7cdc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:54.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.493+0000 7f7ceeffd700 1 -- 172.21.15.43:0/2966684024 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7cf008a720 con 0x7f7cf8157120 2023-06-28T20:08:54.513 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.493+0000 7f7cfe164700 1 --2- 172.21.15.43:0/2966684024 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7cdc06ace0 0x7f7cdc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:54.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.493+0000 7f7cfe164700 1 --2- 172.21.15.43:0/2966684024 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7cdc06ace0 0x7f7cdc06d190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8000c00 tx=0x7f7ce8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:54.514 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.497+0000 7f7ceeffd700 1 -- 172.21.15.43:0/2966684024 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7cf0055c20 con 0x7f7cf8157120 2023-06-28T20:08:54.707 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.696+0000 7f7d003c8700 1 -- 172.21.15.43:0/2966684024 --> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7cf815b3d0 con 0x7f7cdc06ace0 2023-06-28T20:08:54.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.696+0000 7f7ceeffd700 1 -- 172.21.15.43:0/2966684024 <== mgr.4101 v2:172.21.15.43:6832/406144153 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7ce8007030 con 0x7f7cdc06ace0 2023-06-28T20:08:54.709 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.698+0000 7f7d003c8700 1 -- 172.21.15.43:0/2966684024 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7cdc06ace0 msgr2=0x7f7cdc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.698+0000 7f7d003c8700 1 --2- 172.21.15.43:0/2966684024 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7cdc06ace0 0x7f7cdc06d190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8000c00 tx=0x7f7ce8009040 comp rx=0 tx=0).stop 2023-06-28T20:08:54.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.698+0000 7f7d003c8700 1 -- 172.21.15.43:0/2966684024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf8157120 msgr2=0x7f7cf81ef130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.710 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.698+0000 7f7d003c8700 1 --2- 172.21.15.43:0/2966684024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf8157120 0x7f7cf81ef130 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0001520 tx=0x7f7cf0001490 comp rx=0 tx=0).stop 2023-06-28T20:08:54.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.698+0000 7f7d003c8700 1 -- 172.21.15.43:0/2966684024 shutdown_connections 2023-06-28T20:08:54.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.698+0000 7f7d003c8700 1 --2- 172.21.15.43:0/2966684024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cf8157120 0x7f7cf81ef130 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.711 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.698+0000 7f7d003c8700 1 --2- 172.21.15.43:0/2966684024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7ce4004d90 0x7f7cf814b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.698+0000 7f7d003c8700 1 --2- 172.21.15.43:0/2966684024 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f7cdc06ace0 0x7f7cdc06d190 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.698+0000 7f7d003c8700 1 --2- 172.21.15.43:0/2966684024 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f7cf81554b0 0x7f7cf8156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.712 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.698+0000 7f7d003c8700 1 -- 172.21.15.43:0/2966684024 >> 172.21.15.43:0/2966684024 conn(0x7f7cf806b000 msgr2=0x7f7cf814dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:54.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.698+0000 7f7d003c8700 1 -- 172.21.15.43:0/2966684024 shutdown_connections 2023-06-28T20:08:54.713 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.698+0000 7f7d003c8700 1 -- 172.21.15.43:0/2966684024 wait complete. 2023-06-28T20:08:54.713 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 4 to deep-scrub 2023-06-28T20:08:54.715 INFO:tasks.ceph:Scrubbing osd.5 2023-06-28T20:08:54.716 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-06-28T20:08:54.743 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-28T20:08:54.765 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-28T20:08:54.848 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.844+0000 7f3a2cc9f700 1 Processor -- start 2023-06-28T20:08:54.849 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.844+0000 7f3a2cc9f700 1 -- start start 2023-06-28T20:08:54.851 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.845+0000 7f3a2cc9f700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a2806d580 0x7f3a2806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:54.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.845+0000 7f3a2cc9f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3a2806d950 0x7f3a2815c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:54.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.845+0000 7f3a2cc9f700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a2806b710 con 0x7f3a280cfba0 2023-06-28T20:08:54.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.845+0000 7f3a2cc9f700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2806b970 con 0x7f3a2806d950 2023-06-28T20:08:54.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.845+0000 7f3a2cc9f700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2806bc50 con 0x7f3a2806d580 2023-06-28T20:08:54.852 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.845+0000 7f3a26d9d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3a2806d950 0x7f3a2815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:54.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.845+0000 7f3a26d9d700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3a2806d950 0x7f3a2815c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55856/0 (socket says 172.21.15.43:55856) 2023-06-28T20:08:54.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.845+0000 7f3a26d9d700 1 -- 172.21.15.43:0/4127202583 learned_addr learned my addr 172.21.15.43:0/4127202583 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:54.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.846+0000 7f3a2659c700 1 --2- 172.21.15.43:0/4127202583 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a2806d580 0x7f3a2806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:54.853 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.846+0000 7f3a26d9d700 1 -- 172.21.15.43:0/4127202583 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a2806d580 msgr2=0x7f3a2806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:54.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.846+0000 7f3a26d9d700 1 --2- 172.21.15.43:0/4127202583 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a2806d580 0x7f3a2806ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.846+0000 7f3a26d9d700 1 -- 172.21.15.43:0/4127202583 >> v1:172.21.15.178:6789/0 conn(0x7f3a280cfba0 legacy=0x7f3a2806b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.846+0000 7f3a26d9d700 1 -- 172.21.15.43:0/4127202583 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2806bf30 con 0x7f3a2806d950 2023-06-28T20:08:54.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.846+0000 7f3a26d9d700 1 --2- 172.21.15.43:0/4127202583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a2806d950 0x7f3a2815c440 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009e50 tx=0x7f3a2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d1024a62983812 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:54.854 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.846+0000 7f3a2559a700 1 -- 172.21.15.43:0/4127202583 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2000e070 con 0x7f3a2806d950 2023-06-28T20:08:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.846+0000 7f3a2559a700 1 -- 172.21.15.43:0/4127202583 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2000b910 con 0x7f3a2806d950 2023-06-28T20:08:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.846+0000 7f3a2559a700 1 -- 172.21.15.43:0/4127202583 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2000bcc0 con 0x7f3a2806d950 2023-06-28T20:08:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.847+0000 7f3a2cc9f700 1 -- 172.21.15.43:0/4127202583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a2806d950 msgr2=0x7f3a2815c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.847+0000 7f3a2cc9f700 1 --2- 172.21.15.43:0/4127202583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a2806d950 0x7f3a2815c440 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009e50 tx=0x7f3a2000b330 comp rx=0 tx=0).stop 2023-06-28T20:08:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.847+0000 7f3a2cc9f700 1 -- 172.21.15.43:0/4127202583 shutdown_connections 2023-06-28T20:08:54.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.847+0000 7f3a2cc9f700 1 --2- 172.21.15.43:0/4127202583 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a2806d950 0x7f3a2815c440 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.847+0000 7f3a2cc9f700 1 --2- 172.21.15.43:0/4127202583 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a2806d580 0x7f3a2806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.847+0000 7f3a2cc9f700 1 -- 172.21.15.43:0/4127202583 >> 172.21.15.43:0/4127202583 conn(0x7f3a28099f30 msgr2=0x7f3a280d0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:54.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.847+0000 7f3a2cc9f700 1 -- 172.21.15.43:0/4127202583 shutdown_connections 2023-06-28T20:08:54.856 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.847+0000 7f3a2cc9f700 1 -- 172.21.15.43:0/4127202583 wait complete. 2023-06-28T20:08:54.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.848+0000 7f3a2cc9f700 1 Processor -- start 2023-06-28T20:08:54.857 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.848+0000 7f3a2cc9f700 1 -- start start 2023-06-28T20:08:54.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.849+0000 7f3a2cc9f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a2806d580 0x7f3a281f5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:54.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.849+0000 7f3a2cc9f700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a2806d950 0x7f3a281f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:54.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.849+0000 7f3a2cc9f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a280cfba0 0x7f3a2809da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:54.860 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.849+0000 7f3a2cc9f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2815de20 con 0x7f3a280cfba0 2023-06-28T20:08:54.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.849+0000 7f3a2cc9f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a281f5df0 con 0x7f3a2806d580 2023-06-28T20:08:54.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.849+0000 7f3a2cc9f700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2815d7d0 con 0x7f3a2806d950 2023-06-28T20:08:54.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.849+0000 7f3a26d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a280cfba0 0x7f3a2809da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:54.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.849+0000 7f3a26d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a280cfba0 0x7f3a2809da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55862/0 (socket says 172.21.15.43:55862) 2023-06-28T20:08:54.861 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.849+0000 7f3a26d9d700 1 -- 172.21.15.43:0/3155656522 learned_addr learned my addr 172.21.15.43:0/3155656522 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:54.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.849+0000 7f3a2659c700 1 --2- 172.21.15.43:0/3155656522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a2806d580 0x7f3a281f5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:54.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.849+0000 7f3a26d9d700 1 -- 172.21.15.43:0/3155656522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a2806d950 msgr2=0x7f3a281f57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.849+0000 7f3a25d9b700 1 --2- 172.21.15.43:0/3155656522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a2806d950 0x7f3a281f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:54.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.849+0000 7f3a26d9d700 1 --2- 172.21.15.43:0/3155656522 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f3a2806d950 0x7f3a281f57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.862 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.849+0000 7f3a26d9d700 1 -- 172.21.15.43:0/3155656522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a2806d580 msgr2=0x7f3a281f5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.849+0000 7f3a26d9d700 1 --2- 172.21.15.43:0/3155656522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a2806d580 0x7f3a281f5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.849+0000 7f3a26d9d700 1 -- 172.21.15.43:0/3155656522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a20009b30 con 0x7f3a280cfba0 2023-06-28T20:08:54.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.850+0000 7f3a2659c700 1 --2- 172.21.15.43:0/3155656522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a2806d580 0x7f3a281f5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:54.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.850+0000 7f3a26d9d700 1 --2- 172.21.15.43:0/3155656522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a280cfba0 0x7f3a2809da30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3a28067e70 tx=0x7f3a20004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:54.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.850+0000 7f3a16ffd700 1 -- 172.21.15.43:0/3155656522 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a200092e0 con 0x7f3a280cfba0 2023-06-28T20:08:54.863 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.850+0000 7f3a16ffd700 1 -- 172.21.15.43:0/3155656522 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2001c440 con 0x7f3a280cfba0 2023-06-28T20:08:54.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.850+0000 7f3a2cc9f700 1 -- 172.21.15.43:0/3155656522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2815dab0 con 0x7f3a280cfba0 2023-06-28T20:08:54.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.850+0000 7f3a16ffd700 1 -- 172.21.15.43:0/3155656522 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2001c7f0 con 0x7f3a280cfba0 2023-06-28T20:08:54.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.850+0000 7f3a2cc9f700 1 -- 172.21.15.43:0/3155656522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a281f60d0 con 0x7f3a280cfba0 2023-06-28T20:08:54.864 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.851+0000 7f3a2cc9f700 1 -- 172.21.15.43:0/3155656522 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3a2806c4f0 con 0x7f3a280cfba0 2023-06-28T20:08:54.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.851+0000 7f3a16ffd700 1 -- 172.21.15.43:0/3155656522 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3a2001c950 con 0x7f3a280cfba0 2023-06-28T20:08:54.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.852+0000 7f3a16ffd700 1 --2- 172.21.15.43:0/3155656522 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f3a0806ace0 0x7f3a0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:54.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.852+0000 7f3a16ffd700 1 -- 172.21.15.43:0/3155656522 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3a2008b540 con 0x7f3a280cfba0 2023-06-28T20:08:54.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.852+0000 7f3a2659c700 1 --2- 172.21.15.43:0/3155656522 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f3a0806ace0 0x7f3a0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:54.865 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.852+0000 7f3a16ffd700 1 --2- 172.21.15.43:0/3155656522 >> [v2:172.21.15.178:6808/3916491871,v1:172.21.15.178:6809/3916491871] conn(0x7f3a08071150 0x7f3a08073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:54.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.852+0000 7f3a16ffd700 1 -- 172.21.15.43:0/3155656522 --> [v2:172.21.15.178:6808/3916491871,v1:172.21.15.178:6809/3916491871] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3a08073c30 con 0x7f3a08071150 2023-06-28T20:08:54.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.852+0000 7f3a16ffd700 1 -- 172.21.15.43:0/3155656522 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a2008b970 con 0x7f3a280cfba0 2023-06-28T20:08:54.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.852+0000 7f3a2659c700 1 --2- 172.21.15.43:0/3155656522 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f3a0806ace0 0x7f3a0806d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009b40 tx=0x7f3a1c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:54.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.852+0000 7f3a25d9b700 1 --2- 172.21.15.43:0/3155656522 >> [v2:172.21.15.178:6808/3916491871,v1:172.21.15.178:6809/3916491871] conn(0x7f3a08071150 0x7f3a08073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:54.866 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.853+0000 7f3a25d9b700 1 --2- 172.21.15.43:0/3155656522 >> [v2:172.21.15.178:6808/3916491871,v1:172.21.15.178:6809/3916491871] conn(0x7f3a08071150 0x7f3a08073560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:54.867 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.853+0000 7f3a16ffd700 1 -- 172.21.15.43:0/3155656522 <== osd.5 v2:172.21.15.178:6808/3916491871 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3a08073c30 con 0x7f3a08071150 2023-06-28T20:08:54.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.868+0000 7f3a2cc9f700 1 -- 172.21.15.43:0/3155656522 --> [v2:172.21.15.178:6808/3916491871,v1:172.21.15.178:6809/3916491871] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3a2806c210 con 0x7f3a08071150 2023-06-28T20:08:54.871 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.869+0000 7f3a16ffd700 1 -- 172.21.15.43:0/3155656522 <== osd.5 v2:172.21.15.178:6808/3916491871 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f3a180012c0 con 0x7f3a08071150 2023-06-28T20:08:54.872 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-06-28T20:08:54.872 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-28T20:08:54.872 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-06-28T20:08:54.873 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.869+0000 7f3a2cc9f700 1 -- 172.21.15.43:0/3155656522 >> [v2:172.21.15.178:6808/3916491871,v1:172.21.15.178:6809/3916491871] conn(0x7f3a08071150 msgr2=0x7f3a08073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.869+0000 7f3a2cc9f700 1 --2- 172.21.15.43:0/3155656522 >> [v2:172.21.15.178:6808/3916491871,v1:172.21.15.178:6809/3916491871] conn(0x7f3a08071150 0x7f3a08073560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.869+0000 7f3a2cc9f700 1 -- 172.21.15.43:0/3155656522 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f3a0806ace0 msgr2=0x7f3a0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.869+0000 7f3a2cc9f700 1 --2- 172.21.15.43:0/3155656522 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f3a0806ace0 0x7f3a0806d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009b40 tx=0x7f3a1c008040 comp rx=0 tx=0).stop 2023-06-28T20:08:54.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.869+0000 7f3a2cc9f700 1 -- 172.21.15.43:0/3155656522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a280cfba0 msgr2=0x7f3a2809da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:54.874 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.869+0000 7f3a2cc9f700 1 --2- 172.21.15.43:0/3155656522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a280cfba0 0x7f3a2809da30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3a28067e70 tx=0x7f3a20004ad0 comp rx=0 tx=0).stop 2023-06-28T20:08:54.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.869+0000 7f3a26d9d700 1 -- 172.21.15.43:0/3155656522 reap_dead start 2023-06-28T20:08:54.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.869+0000 7f3a2cc9f700 1 -- 172.21.15.43:0/3155656522 shutdown_connections 2023-06-28T20:08:54.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.869+0000 7f3a2cc9f700 1 -- 172.21.15.43:0/3155656522 >> 172.21.15.43:0/3155656522 conn(0x7f3a28099f30 msgr2=0x7f3a2815af60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:54.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.869+0000 7f3a2cc9f700 1 -- 172.21.15.43:0/3155656522 shutdown_connections 2023-06-28T20:08:54.875 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:54.869+0000 7f3a2cc9f700 1 -- 172.21.15.43:0/3155656522 wait complete. 2023-06-28T20:08:54.884 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-06-28T20:08:55.050 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.047+0000 7fb06a2c9700 1 Processor -- start 2023-06-28T20:08:55.051 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.048+0000 7fb06a2c9700 1 -- start start 2023-06-28T20:08:55.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.048+0000 7fb06a2c9700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0641592d0 0x7fb0640b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.053 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.048+0000 7fb06a2c9700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb0641596a0 0x7fb0640b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.048+0000 7fb06a2c9700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb064070b40 con 0x7fb064071900 2023-06-28T20:08:55.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.048+0000 7fb06a2c9700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb064070e20 con 0x7fb0641596a0 2023-06-28T20:08:55.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.048+0000 7fb06a2c9700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb064071100 con 0x7fb0641592d0 2023-06-28T20:08:55.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.049+0000 7fb0637fe700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb0641596a0 0x7fb0640b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.054 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.049+0000 7fb068866700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb064071900 0x7fb06415d070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45058/0 (socket says 172.21.15.43:45058) 2023-06-28T20:08:55.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.049+0000 7fb068866700 1 -- 172.21.15.43:0/425882753 learned_addr learned my addr 172.21.15.43:0/425882753 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:55.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.049+0000 7fb063fff700 1 --2- 172.21.15.43:0/425882753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0641592d0 0x7fb0640b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.049+0000 7fb0637fe700 1 -- 172.21.15.43:0/425882753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0641592d0 msgr2=0x7fb0640b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:55.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.049+0000 7fb0637fe700 1 --2- 172.21.15.43:0/425882753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0641592d0 0x7fb0640b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.055 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.049+0000 7fb0637fe700 1 -- 172.21.15.43:0/425882753 >> v1:172.21.15.43:6789/0 conn(0x7fb064071900 legacy=0x7fb06415d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.049+0000 7fb062ffd700 1 -- 172.21.15.43:0/425882753 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4072765137 0 0) 0x7fb064070b40 con 0x7fb064071900 2023-06-28T20:08:55.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.049+0000 7fb0637fe700 1 -- 172.21.15.43:0/425882753 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0640713e0 con 0x7fb0641596a0 2023-06-28T20:08:55.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.049+0000 7fb0637fe700 1 --2- 172.21.15.43:0/425882753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0641596a0 0x7fb0640b1f10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009840 tx=0x7fb04c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb59659366db4f6 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:55.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.049+0000 7fb062ffd700 1 -- 172.21.15.43:0/425882753 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c011830 con 0x7fb0641596a0 2023-06-28T20:08:55.056 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.049+0000 7fb062ffd700 1 -- 172.21.15.43:0/425882753 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04c011990 con 0x7fb0641596a0 2023-06-28T20:08:55.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.049+0000 7fb062ffd700 1 -- 172.21.15.43:0/425882753 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04c011d20 con 0x7fb0641596a0 2023-06-28T20:08:55.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.050+0000 7fb06a2c9700 1 -- 172.21.15.43:0/425882753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0641596a0 msgr2=0x7fb0640b1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.050+0000 7fb06a2c9700 1 --2- 172.21.15.43:0/425882753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0641596a0 0x7fb0640b1f10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009840 tx=0x7fb04c009b50 comp rx=0 tx=0).stop 2023-06-28T20:08:55.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.050+0000 7fb06a2c9700 1 -- 172.21.15.43:0/425882753 shutdown_connections 2023-06-28T20:08:55.057 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.050+0000 7fb06a2c9700 1 --2- 172.21.15.43:0/425882753 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0641592d0 0x7fb0640b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.050+0000 7fb06a2c9700 1 --2- 172.21.15.43:0/425882753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb0641596a0 0x7fb0640b1f10 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.050+0000 7fb06a2c9700 1 -- 172.21.15.43:0/425882753 >> 172.21.15.43:0/425882753 conn(0x7fb06406b000 msgr2=0x7fb06406b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:55.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.050+0000 7fb06a2c9700 1 -- 172.21.15.43:0/425882753 shutdown_connections 2023-06-28T20:08:55.058 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.050+0000 7fb06a2c9700 1 -- 172.21.15.43:0/425882753 wait complete. 2023-06-28T20:08:55.060 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.051+0000 7fb06a2c9700 1 Processor -- start 2023-06-28T20:08:55.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.051+0000 7fb06a2c9700 1 -- start start 2023-06-28T20:08:55.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.052+0000 7fb06a2c9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064071900 0x7fb06414cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.052+0000 7fb06a2c9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0641592d0 0x7fb06415a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.061 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.052+0000 7fb06a2c9700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0641596a0 0x7fb064150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.052+0000 7fb06a2c9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb064151d30 con 0x7fb064071900 2023-06-28T20:08:55.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.052+0000 7fb06a2c9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb064151770 con 0x7fb0641592d0 2023-06-28T20:08:55.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.052+0000 7fb06a2c9700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb064151a50 con 0x7fb0641596a0 2023-06-28T20:08:55.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.052+0000 7fb063fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064071900 0x7fb06414cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.062 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.052+0000 7fb063fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064071900 0x7fb06414cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55874/0 (socket says 172.21.15.43:55874) 2023-06-28T20:08:55.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.052+0000 7fb063fff700 1 -- 172.21.15.43:0/1044137074 learned_addr learned my addr 172.21.15.43:0/1044137074 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:55.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.052+0000 7fb0637fe700 1 --2- 172.21.15.43:0/1044137074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0641592d0 0x7fb06415a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.052+0000 7fb068866700 1 --2- 172.21.15.43:0/1044137074 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0641596a0 0x7fb064150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.052+0000 7fb063fff700 1 -- 172.21.15.43:0/1044137074 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0641596a0 msgr2=0x7fb064150d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.063 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.052+0000 7fb063fff700 1 --2- 172.21.15.43:0/1044137074 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0641596a0 0x7fb064150d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.052+0000 7fb063fff700 1 -- 172.21.15.43:0/1044137074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0641592d0 msgr2=0x7fb06415a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.052+0000 7fb063fff700 1 --2- 172.21.15.43:0/1044137074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0641592d0 0x7fb06415a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.052+0000 7fb063fff700 1 -- 172.21.15.43:0/1044137074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c0094a0 con 0x7fb064071900 2023-06-28T20:08:55.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.052+0000 7fb068866700 1 --2- 172.21.15.43:0/1044137074 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0641596a0 0x7fb064150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:08:55.064 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.053+0000 7fb063fff700 1 --2- 172.21.15.43:0/1044137074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064071900 0x7fb06414cff0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb05400e810 tx=0x7fb05400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:55.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.053+0000 7fb060ff9700 1 -- 172.21.15.43:0/1044137074 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb054013610 con 0x7fb064071900 2023-06-28T20:08:55.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.053+0000 7fb060ff9700 1 -- 172.21.15.43:0/1044137074 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb054013770 con 0x7fb064071900 2023-06-28T20:08:55.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.053+0000 7fb06a2c9700 1 -- 172.21.15.43:0/1044137074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb064159c10 con 0x7fb064071900 2023-06-28T20:08:55.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.053+0000 7fb060ff9700 1 -- 172.21.15.43:0/1044137074 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb054013b20 con 0x7fb064071900 2023-06-28T20:08:55.065 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.053+0000 7fb06a2c9700 1 -- 172.21.15.43:0/1044137074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb06414d530 con 0x7fb064071900 2023-06-28T20:08:55.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.054+0000 7fb06a2c9700 1 -- 172.21.15.43:0/1044137074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb06415a260 con 0x7fb064071900 2023-06-28T20:08:55.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.054+0000 7fb060ff9700 1 -- 172.21.15.43:0/1044137074 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb054013c80 con 0x7fb064071900 2023-06-28T20:08:55.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.055+0000 7fb060ff9700 1 --2- 172.21.15.43:0/1044137074 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb04406ace0 0x7fb04406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.055+0000 7fb060ff9700 1 -- 172.21.15.43:0/1044137074 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fb054088dd0 con 0x7fb064071900 2023-06-28T20:08:55.066 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.055+0000 7fb0637fe700 1 --2- 172.21.15.43:0/1044137074 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb04406ace0 0x7fb04406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.055+0000 7fb0637fe700 1 --2- 172.21.15.43:0/1044137074 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb04406ace0 0x7fb04406d190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb064159ab0 tx=0x7fb04c01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:55.067 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.059+0000 7fb060ff9700 1 -- 172.21.15.43:0/1044137074 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb054057c20 con 0x7fb064071900 2023-06-28T20:08:55.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.257+0000 7fb06a2c9700 1 -- 172.21.15.43:0/1044137074 --> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fb064151bc0 con 0x7fb04406ace0 2023-06-28T20:08:55.260 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.257+0000 7fb060ff9700 1 -- 172.21.15.43:0/1044137074 <== mgr.4101 v2:172.21.15.43:6832/406144153 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb064151bc0 con 0x7fb04406ace0 2023-06-28T20:08:55.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.259+0000 7fb06a2c9700 1 -- 172.21.15.43:0/1044137074 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb04406ace0 msgr2=0x7fb04406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.262 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.259+0000 7fb06a2c9700 1 --2- 172.21.15.43:0/1044137074 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb04406ace0 0x7fb04406d190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb064159ab0 tx=0x7fb04c01c040 comp rx=0 tx=0).stop 2023-06-28T20:08:55.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.259+0000 7fb06a2c9700 1 -- 172.21.15.43:0/1044137074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064071900 msgr2=0x7fb06414cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.259+0000 7fb06a2c9700 1 --2- 172.21.15.43:0/1044137074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064071900 0x7fb06414cff0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb05400e810 tx=0x7fb05400eb20 comp rx=0 tx=0).stop 2023-06-28T20:08:55.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.259+0000 7fb06a2c9700 1 -- 172.21.15.43:0/1044137074 shutdown_connections 2023-06-28T20:08:55.263 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.259+0000 7fb06a2c9700 1 --2- 172.21.15.43:0/1044137074 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fb04406ace0 0x7fb04406d190 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.259+0000 7fb06a2c9700 1 --2- 172.21.15.43:0/1044137074 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fb0641596a0 0x7fb064150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.259+0000 7fb06a2c9700 1 --2- 172.21.15.43:0/1044137074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb064071900 0x7fb06414cff0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.259+0000 7fb06a2c9700 1 --2- 172.21.15.43:0/1044137074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0641592d0 0x7fb06415a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.259+0000 7fb06a2c9700 1 -- 172.21.15.43:0/1044137074 >> 172.21.15.43:0/1044137074 conn(0x7fb06406b000 msgr2=0x7fb06406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:55.264 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.259+0000 7fb06a2c9700 1 -- 172.21.15.43:0/1044137074 shutdown_connections 2023-06-28T20:08:55.265 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.259+0000 7fb06a2c9700 1 -- 172.21.15.43:0/1044137074 wait complete. 2023-06-28T20:08:55.265 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 5 to deep-scrub 2023-06-28T20:08:55.275 INFO:tasks.ceph:Scrubbing osd.6 2023-06-28T20:08:55.275 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-06-28T20:08:55.405 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.401+0000 7ffa8fbbb700 1 Processor -- start 2023-06-28T20:08:55.406 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.402+0000 7ffa8fbbb700 1 -- start start 2023-06-28T20:08:55.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.402+0000 7ffa8fbbb700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffa880cee80 0x7ffa880ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.408 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.402+0000 7ffa8fbbb700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa8815a9f0 con 0x7ffa880b01b0 2023-06-28T20:08:55.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.402+0000 7ffa8fbbb700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa880725e0 con 0x7ffa8815a620 2023-06-28T20:08:55.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.402+0000 7ffa8fbbb700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa8806e6a0 con 0x7ffa880cee80 2023-06-28T20:08:55.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.403+0000 7ffa8d156700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffa880cee80 0x7ffa880ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.409 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.403+0000 7ffa8e158700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ffa880b01b0 0x7ffa880aec30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45072/0 (socket says 172.21.15.43:45072) 2023-06-28T20:08:55.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.403+0000 7ffa8e158700 1 -- 172.21.15.43:0/3517014125 learned_addr learned my addr 172.21.15.43:0/3517014125 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:55.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.403+0000 7ffa8d156700 1 -- 172.21.15.43:0/3517014125 >> v1:172.21.15.178:6789/0 conn(0x7ffa8815a620 legacy=0x7ffa880ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:55.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.403+0000 7ffa8d156700 1 -- 172.21.15.43:0/3517014125 >> v1:172.21.15.43:6789/0 conn(0x7ffa880b01b0 legacy=0x7ffa880aec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.403+0000 7ffa8d156700 1 -- 172.21.15.43:0/3517014125 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa880ad930 con 0x7ffa880cee80 2023-06-28T20:08:55.410 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.403+0000 7ffa8c955700 1 -- 172.21.15.43:0/3517014125 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 221182433 0 0) 0x7ffa8815a9f0 con 0x7ffa880b01b0 2023-06-28T20:08:55.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.403+0000 7ffa8d156700 1 --2- 172.21.15.43:0/3517014125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa880cee80 0x7ffa880ae6f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ffa84009840 tx=0x7ffa84009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb1bb415c336a11 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:55.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.403+0000 7ffa8c955700 1 -- 172.21.15.43:0/3517014125 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa84004230 con 0x7ffa880cee80 2023-06-28T20:08:55.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.403+0000 7ffa8c955700 1 -- 172.21.15.43:0/3517014125 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa84004390 con 0x7ffa880cee80 2023-06-28T20:08:55.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.403+0000 7ffa8c955700 1 -- 172.21.15.43:0/3517014125 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa840036a0 con 0x7ffa880cee80 2023-06-28T20:08:55.411 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.404+0000 7ffa8fbbb700 1 -- 172.21.15.43:0/3517014125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa880cee80 msgr2=0x7ffa880ae6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.404+0000 7ffa8fbbb700 1 --2- 172.21.15.43:0/3517014125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa880cee80 0x7ffa880ae6f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ffa84009840 tx=0x7ffa84009b50 comp rx=0 tx=0).stop 2023-06-28T20:08:55.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.404+0000 7ffa8fbbb700 1 -- 172.21.15.43:0/3517014125 shutdown_connections 2023-06-28T20:08:55.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.404+0000 7ffa8fbbb700 1 --2- 172.21.15.43:0/3517014125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa880cee80 0x7ffa880ae6f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.412 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.404+0000 7ffa8fbbb700 1 -- 172.21.15.43:0/3517014125 >> 172.21.15.43:0/3517014125 conn(0x7ffa88099f30 msgr2=0x7ffa880d0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:55.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.404+0000 7ffa8fbbb700 1 -- 172.21.15.43:0/3517014125 shutdown_connections 2023-06-28T20:08:55.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.404+0000 7ffa8fbbb700 1 -- 172.21.15.43:0/3517014125 wait complete. 2023-06-28T20:08:55.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.405+0000 7ffa8fbbb700 1 Processor -- start 2023-06-28T20:08:55.413 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.405+0000 7ffa8fbbb700 1 -- start start 2023-06-28T20:08:55.415 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.406+0000 7ffa8fbbb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa880b01b0 0x7ffa880c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.406+0000 7ffa8fbbb700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa880cee80 0x7ffa880ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.406+0000 7ffa8fbbb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa8815a620 0x7ffa880c37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.416 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.406+0000 7ffa8fbbb700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa88156e70 con 0x7ffa8815a620 2023-06-28T20:08:55.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.406+0000 7ffa8fbbb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa880c70c0 con 0x7ffa880b01b0 2023-06-28T20:08:55.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.406+0000 7ffa8fbbb700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa880c6130 con 0x7ffa880cee80 2023-06-28T20:08:55.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.406+0000 7ffa8e158700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa8815a620 0x7ffa880c37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.417 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.406+0000 7ffa8e158700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa8815a620 0x7ffa880c37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55892/0 (socket says 172.21.15.43:55892) 2023-06-28T20:08:55.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.406+0000 7ffa8e158700 1 -- 172.21.15.43:0/2403515621 learned_addr learned my addr 172.21.15.43:0/2403515621 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:55.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.406+0000 7ffa8e158700 1 -- 172.21.15.43:0/2403515621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa880cee80 msgr2=0x7ffa880ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.406+0000 7ffa8e158700 1 --2- 172.21.15.43:0/2403515621 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7ffa880cee80 0x7ffa880ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.418 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.406+0000 7ffa8e158700 1 -- 172.21.15.43:0/2403515621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa880b01b0 msgr2=0x7ffa880c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.406+0000 7ffa8e158700 1 --2- 172.21.15.43:0/2403515621 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa880b01b0 0x7ffa880c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.406+0000 7ffa8e158700 1 -- 172.21.15.43:0/2403515621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa840094a0 con 0x7ffa8815a620 2023-06-28T20:08:55.419 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.406+0000 7ffa8e158700 1 --2- 172.21.15.43:0/2403515621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa8815a620 0x7ffa880c37a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000cc70 tx=0x7ffa8000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:55.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.407+0000 7ffa7a7fc700 1 -- 172.21.15.43:0/2403515621 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8000ad80 con 0x7ffa8815a620 2023-06-28T20:08:55.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.407+0000 7ffa7a7fc700 1 -- 172.21.15.43:0/2403515621 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8000ce70 con 0x7ffa8815a620 2023-06-28T20:08:55.420 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.407+0000 7ffa8fbbb700 1 -- 172.21.15.43:0/2403515621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa8806c9d0 con 0x7ffa8815a620 2023-06-28T20:08:55.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.407+0000 7ffa7a7fc700 1 -- 172.21.15.43:0/2403515621 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa800072f0 con 0x7ffa8815a620 2023-06-28T20:08:55.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.407+0000 7ffa8fbbb700 1 -- 172.21.15.43:0/2403515621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa880c6660 con 0x7ffa8815a620 2023-06-28T20:08:55.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.407+0000 7ffa8fbbb700 1 -- 172.21.15.43:0/2403515621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffa8806cdc0 con 0x7ffa8815a620 2023-06-28T20:08:55.421 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.408+0000 7ffa7a7fc700 1 -- 172.21.15.43:0/2403515621 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffa80007450 con 0x7ffa8815a620 2023-06-28T20:08:55.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.408+0000 7ffa7a7fc700 1 --2- 172.21.15.43:0/2403515621 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ffa6c06ace0 0x7ffa6c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.409+0000 7ffa7a7fc700 1 -- 172.21.15.43:0/2403515621 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ffa80089670 con 0x7ffa8815a620 2023-06-28T20:08:55.422 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.409+0000 7ffa8d957700 1 --2- 172.21.15.43:0/2403515621 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ffa6c06ace0 0x7ffa6c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.409+0000 7ffa7a7fc700 1 --2- 172.21.15.43:0/2403515621 >> [v2:172.21.15.178:6800/2113985646,v1:172.21.15.178:6801/2113985646] conn(0x7ffa6c071150 0x7ffa6c073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.409+0000 7ffa7a7fc700 1 -- 172.21.15.43:0/2403515621 --> [v2:172.21.15.178:6800/2113985646,v1:172.21.15.178:6801/2113985646] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa6c073c30 con 0x7ffa6c071150 2023-06-28T20:08:55.423 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.409+0000 7ffa7a7fc700 1 -- 172.21.15.43:0/2403515621 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa80089aa0 con 0x7ffa8815a620 2023-06-28T20:08:55.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.409+0000 7ffa8d957700 1 --2- 172.21.15.43:0/2403515621 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ffa6c06ace0 0x7ffa6c06d190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0096f0 tx=0x7ffa7c006c40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:55.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.409+0000 7ffa8d156700 1 --2- 172.21.15.43:0/2403515621 >> [v2:172.21.15.178:6800/2113985646,v1:172.21.15.178:6801/2113985646] conn(0x7ffa6c071150 0x7ffa6c073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.409+0000 7ffa8d156700 1 --2- 172.21.15.43:0/2403515621 >> [v2:172.21.15.178:6800/2113985646,v1:172.21.15.178:6801/2113985646] conn(0x7ffa6c071150 0x7ffa6c073560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:55.424 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.410+0000 7ffa7a7fc700 1 -- 172.21.15.43:0/2403515621 <== osd.6 v2:172.21.15.178:6800/2113985646 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ffa6c073c30 con 0x7ffa6c071150 2023-06-28T20:08:55.427 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.425+0000 7ffa8fbbb700 1 -- 172.21.15.43:0/2403515621 --> [v2:172.21.15.178:6800/2113985646,v1:172.21.15.178:6801/2113985646] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ffa880adef0 con 0x7ffa6c071150 2023-06-28T20:08:55.428 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.425+0000 7ffa7a7fc700 1 -- 172.21.15.43:0/2403515621 <== osd.6 v2:172.21.15.178:6800/2113985646 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ffa84006c60 con 0x7ffa6c071150 2023-06-28T20:08:55.429 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-06-28T20:08:55.429 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-28T20:08:55.430 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-06-28T20:08:55.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.425+0000 7ffa8fbbb700 1 -- 172.21.15.43:0/2403515621 >> [v2:172.21.15.178:6800/2113985646,v1:172.21.15.178:6801/2113985646] conn(0x7ffa6c071150 msgr2=0x7ffa6c073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.431 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.425+0000 7ffa8fbbb700 1 --2- 172.21.15.43:0/2403515621 >> [v2:172.21.15.178:6800/2113985646,v1:172.21.15.178:6801/2113985646] conn(0x7ffa6c071150 0x7ffa6c073560 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.425+0000 7ffa8fbbb700 1 -- 172.21.15.43:0/2403515621 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ffa6c06ace0 msgr2=0x7ffa6c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.425+0000 7ffa8fbbb700 1 --2- 172.21.15.43:0/2403515621 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7ffa6c06ace0 0x7ffa6c06d190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0096f0 tx=0x7ffa7c006c40 comp rx=0 tx=0).stop 2023-06-28T20:08:55.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.425+0000 7ffa8fbbb700 1 -- 172.21.15.43:0/2403515621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa8815a620 msgr2=0x7ffa880c37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.432 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.425+0000 7ffa8fbbb700 1 --2- 172.21.15.43:0/2403515621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa8815a620 0x7ffa880c37a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000cc70 tx=0x7ffa8000a340 comp rx=0 tx=0).stop 2023-06-28T20:08:55.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.426+0000 7ffa8e158700 1 -- 172.21.15.43:0/2403515621 reap_dead start 2023-06-28T20:08:55.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.426+0000 7ffa8fbbb700 1 -- 172.21.15.43:0/2403515621 shutdown_connections 2023-06-28T20:08:55.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.426+0000 7ffa8fbbb700 1 -- 172.21.15.43:0/2403515621 >> 172.21.15.43:0/2403515621 conn(0x7ffa88099f30 msgr2=0x7ffa881594b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:55.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.426+0000 7ffa8fbbb700 1 -- 172.21.15.43:0/2403515621 shutdown_connections 2023-06-28T20:08:55.433 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.426+0000 7ffa8fbbb700 1 -- 172.21.15.43:0/2403515621 wait complete. 2023-06-28T20:08:55.440 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-06-28T20:08:55.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.564+0000 7f391039a700 1 Processor -- start 2023-06-28T20:08:55.567 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.564+0000 7f391039a700 1 -- start start 2023-06-28T20:08:55.569 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.565+0000 7f391039a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f390815a180 0x7f39080ac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.565+0000 7f391039a700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f39080cfe20 0x7f390815d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.565+0000 7f391039a700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f390815d990 0x7f390806c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.570 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.565+0000 7f391039a700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f390815cce0 con 0x7f390815d990 2023-06-28T20:08:55.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.565+0000 7f391039a700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39080ae770 con 0x7f39080cfe20 2023-06-28T20:08:55.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.565+0000 7f391039a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39080ad0a0 con 0x7f390815a180 2023-06-28T20:08:55.571 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.565+0000 7f390e937700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f390815d990 0x7f390806c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.565+0000 7f390e937700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f390815d990 0x7f390806c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55900/0 (socket says 172.21.15.43:55900) 2023-06-28T20:08:55.572 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.565+0000 7f390e937700 1 -- 172.21.15.43:0/3632855758 learned_addr learned my addr 172.21.15.43:0/3632855758 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:55.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.565+0000 7f390d935700 1 --2- 172.21.15.43:0/3632855758 >> v2:172.21.15.178:3300/0 conn(0x7f39080cfe20 0x7f390815d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.565+0000 7f390e136700 1 --2- 172.21.15.43:0/3632855758 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f390815a180 0x7f39080ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.565+0000 7f390e937700 1 -- 172.21.15.43:0/3632855758 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f390815a180 msgr2=0x7f39080ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:55.573 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.565+0000 7f390e937700 1 --2- 172.21.15.43:0/3632855758 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f390815a180 0x7f39080ac870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.565+0000 7f390e937700 1 -- 172.21.15.43:0/3632855758 >> v2:172.21.15.178:3300/0 conn(0x7f39080cfe20 msgr2=0x7f390815d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:55.574 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.565+0000 7f390e937700 1 --2- 172.21.15.43:0/3632855758 >> v2:172.21.15.178:3300/0 conn(0x7f39080cfe20 0x7f390815d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.565+0000 7f390e937700 1 -- 172.21.15.43:0/3632855758 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39080ad380 con 0x7f390815d990 2023-06-28T20:08:55.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.565+0000 7f390e136700 1 --2- 172.21.15.43:0/3632855758 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f390815a180 0x7f39080ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:55.575 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.565+0000 7f390e937700 1 --2- 172.21.15.43:0/3632855758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390815d990 0x7f390806c0a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f390806fd80 tx=0x7f390000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1186a9b179c33570 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:55.576 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.565+0000 7f390d134700 1 -- 172.21.15.43:0/3632855758 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390000e070 con 0x7f390815d990 2023-06-28T20:08:55.577 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.566+0000 7f390d134700 1 -- 172.21.15.43:0/3632855758 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390000b9c0 con 0x7f390815d990 2023-06-28T20:08:55.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.566+0000 7f390d134700 1 -- 172.21.15.43:0/3632855758 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390000bd70 con 0x7f390815d990 2023-06-28T20:08:55.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.566+0000 7f391039a700 1 -- 172.21.15.43:0/3632855758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390815d990 msgr2=0x7f390806c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.566+0000 7f391039a700 1 --2- 172.21.15.43:0/3632855758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390815d990 0x7f390806c0a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f390806fd80 tx=0x7f390000b330 comp rx=0 tx=0).stop 2023-06-28T20:08:55.578 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.566+0000 7f391039a700 1 -- 172.21.15.43:0/3632855758 shutdown_connections 2023-06-28T20:08:55.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.566+0000 7f391039a700 1 --2- 172.21.15.43:0/3632855758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390815d990 0x7f390806c0a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.566+0000 7f391039a700 1 --2- 172.21.15.43:0/3632855758 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f390815a180 0x7f39080ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.566+0000 7f391039a700 1 --2- 172.21.15.43:0/3632855758 >> v2:172.21.15.178:3300/0 conn(0x7f39080cfe20 0x7f390815d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.566+0000 7f391039a700 1 -- 172.21.15.43:0/3632855758 >> 172.21.15.43:0/3632855758 conn(0x7f39080b1a50 msgr2=0x7f39080cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:55.579 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.566+0000 7f391039a700 1 -- 172.21.15.43:0/3632855758 shutdown_connections 2023-06-28T20:08:55.580 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.566+0000 7f391039a700 1 -- 172.21.15.43:0/3632855758 wait complete. 2023-06-28T20:08:55.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.567+0000 7f391039a700 1 Processor -- start 2023-06-28T20:08:55.582 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.567+0000 7f391039a700 1 -- start start 2023-06-28T20:08:55.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.568+0000 7f391039a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39080cfe20 0x7f39080c2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.568+0000 7f391039a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390815a180 0x7f39080b3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.568+0000 7f391039a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39080b20e0 0x7f390815bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.568+0000 7f391039a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39080c3d80 con 0x7f390815a180 2023-06-28T20:08:55.583 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.568+0000 7f391039a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39080c4800 con 0x7f39080b20e0 2023-06-28T20:08:55.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.568+0000 7f391039a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39080b2c10 con 0x7f39080cfe20 2023-06-28T20:08:55.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.568+0000 7f390d935700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390815a180 0x7f39080b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.568+0000 7f390d935700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390815a180 0x7f39080b3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55904/0 (socket says 172.21.15.43:55904) 2023-06-28T20:08:55.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.568+0000 7f390d935700 1 -- 172.21.15.43:0/4242317489 learned_addr learned my addr 172.21.15.43:0/4242317489 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:55.584 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.568+0000 7f390e937700 1 --2- 172.21.15.43:0/4242317489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39080b20e0 0x7f390815bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.568+0000 7f390e136700 1 --2- 172.21.15.43:0/4242317489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39080cfe20 0x7f39080c2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.568+0000 7f390d935700 1 -- 172.21.15.43:0/4242317489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39080cfe20 msgr2=0x7f39080c2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.568+0000 7f390d935700 1 --2- 172.21.15.43:0/4242317489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39080cfe20 0x7f39080c2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.585 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.569+0000 7f390d935700 1 -- 172.21.15.43:0/4242317489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39080b20e0 msgr2=0x7f390815bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.569+0000 7f390d935700 1 --2- 172.21.15.43:0/4242317489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39080b20e0 0x7f390815bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.569+0000 7f390d935700 1 -- 172.21.15.43:0/4242317489 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3900009b30 con 0x7f390815a180 2023-06-28T20:08:55.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.569+0000 7f390d935700 1 --2- 172.21.15.43:0/4242317489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390815a180 0x7f39080b3310 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f38f800d830 tx=0x7f38f800db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:55.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.569+0000 7f38feffd700 1 -- 172.21.15.43:0/4242317489 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f80125f0 con 0x7f390815a180 2023-06-28T20:08:55.586 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.569+0000 7f38feffd700 1 -- 172.21.15.43:0/4242317489 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38f8012750 con 0x7f390815a180 2023-06-28T20:08:55.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.569+0000 7f391039a700 1 -- 172.21.15.43:0/4242317489 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39080c34e0 con 0x7f390815a180 2023-06-28T20:08:55.587 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.569+0000 7f38feffd700 1 -- 172.21.15.43:0/4242317489 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f8012ae0 con 0x7f390815a180 2023-06-28T20:08:55.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.569+0000 7f391039a700 1 -- 172.21.15.43:0/4242317489 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39080c3aa0 con 0x7f390815a180 2023-06-28T20:08:55.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.570+0000 7f391039a700 1 -- 172.21.15.43:0/4242317489 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39080728b0 con 0x7f390815a180 2023-06-28T20:08:55.588 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.570+0000 7f38feffd700 1 -- 172.21.15.43:0/4242317489 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f38f8012c40 con 0x7f390815a180 2023-06-28T20:08:55.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.571+0000 7f38feffd700 1 --2- 172.21.15.43:0/4242317489 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f38ec06ace0 0x7f38ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.571+0000 7f38feffd700 1 -- 172.21.15.43:0/4242317489 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f38f80891d0 con 0x7f390815a180 2023-06-28T20:08:55.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.571+0000 7f390e136700 1 --2- 172.21.15.43:0/4242317489 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f38ec06ace0 0x7f38ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.571+0000 7f390e136700 1 --2- 172.21.15.43:0/4242317489 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f38ec06ace0 0x7f38ec06d190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3904006fd0 tx=0x7f3904008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:55.589 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.575+0000 7f38feffd700 1 -- 172.21.15.43:0/4242317489 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f38f8054590 con 0x7f390815a180 2023-06-28T20:08:55.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.772+0000 7f391039a700 1 -- 172.21.15.43:0/4242317489 --> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f39080acdb0 con 0x7f38ec06ace0 2023-06-28T20:08:55.775 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.772+0000 7f38feffd700 1 -- 172.21.15.43:0/4242317489 <== mgr.4101 v2:172.21.15.43:6832/406144153 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f390400c050 con 0x7f38ec06ace0 2023-06-28T20:08:55.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.774+0000 7f391039a700 1 -- 172.21.15.43:0/4242317489 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f38ec06ace0 msgr2=0x7f38ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.777 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.774+0000 7f391039a700 1 --2- 172.21.15.43:0/4242317489 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f38ec06ace0 0x7f38ec06d190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3904006fd0 tx=0x7f3904008040 comp rx=0 tx=0).stop 2023-06-28T20:08:55.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.774+0000 7f391039a700 1 -- 172.21.15.43:0/4242317489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390815a180 msgr2=0x7f39080b3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.774+0000 7f391039a700 1 --2- 172.21.15.43:0/4242317489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390815a180 0x7f39080b3310 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f38f800d830 tx=0x7f38f800db40 comp rx=0 tx=0).stop 2023-06-28T20:08:55.778 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.774+0000 7f391039a700 1 -- 172.21.15.43:0/4242317489 shutdown_connections 2023-06-28T20:08:55.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.774+0000 7f391039a700 1 --2- 172.21.15.43:0/4242317489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39080b20e0 0x7f390815bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.774+0000 7f391039a700 1 --2- 172.21.15.43:0/4242317489 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f38ec06ace0 0x7f38ec06d190 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.779 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.774+0000 7f391039a700 1 --2- 172.21.15.43:0/4242317489 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f39080cfe20 0x7f39080c2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.774+0000 7f391039a700 1 --2- 172.21.15.43:0/4242317489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f390815a180 0x7f39080b3310 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.774+0000 7f391039a700 1 -- 172.21.15.43:0/4242317489 >> 172.21.15.43:0/4242317489 conn(0x7f39080b1a50 msgr2=0x7f39081596e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:55.780 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.774+0000 7f391039a700 1 -- 172.21.15.43:0/4242317489 shutdown_connections 2023-06-28T20:08:55.781 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.774+0000 7f391039a700 1 -- 172.21.15.43:0/4242317489 wait complete. 2023-06-28T20:08:55.781 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 6 to deep-scrub 2023-06-28T20:08:55.789 INFO:tasks.ceph:Scrubbing osd.7 2023-06-28T20:08:55.790 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-06-28T20:08:55.919 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.916+0000 7f1caf75e700 1 Processor -- start 2023-06-28T20:08:55.920 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.917+0000 7f1caf75e700 1 -- start start 2023-06-28T20:08:55.923 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.918+0000 7f1caf75e700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1ca806a600 0x7f1ca80cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.918+0000 7f1caf75e700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f1ca806f870 0x7f1ca80ae800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.918+0000 7f1caf75e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca80720f0 con 0x7f1ca8158dd0 2023-06-28T20:08:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.918+0000 7f1caf75e700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca8071b70 con 0x7f1ca806a600 2023-06-28T20:08:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.918+0000 7f1caf75e700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca806fc70 con 0x7f1ca806f870 2023-06-28T20:08:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.918+0000 7f1caccf9700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1ca806a600 0x7f1ca80cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.918+0000 7f1cad4fa700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1ca8158dd0 0x7f1ca8069ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45096/0 (socket says 172.21.15.43:45096) 2023-06-28T20:08:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.918+0000 7f1cad4fa700 1 -- 172.21.15.43:0/3889877571 learned_addr learned my addr 172.21.15.43:0/3889877571 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.918+0000 7f1cadcfb700 1 --2- 172.21.15.43:0/3889877571 >> v2:172.21.15.178:3300/0 conn(0x7f1ca806f870 0x7f1ca80ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.918+0000 7f1c9ffff700 1 -- 172.21.15.43:0/3889877571 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3537554460 0 0) 0x7f1ca80720f0 con 0x7f1ca8158dd0 2023-06-28T20:08:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.918+0000 7f1c9ffff700 1 -- 172.21.15.43:0/3889877571 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c940035b0 con 0x7f1ca8158dd0 2023-06-28T20:08:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.918+0000 7f1caccf9700 1 -- 172.21.15.43:0/3889877571 >> v2:172.21.15.178:3300/0 conn(0x7f1ca806f870 msgr2=0x7f1ca80ae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.918+0000 7f1caccf9700 1 --2- 172.21.15.43:0/3889877571 >> v2:172.21.15.178:3300/0 conn(0x7f1ca806f870 0x7f1ca80ae800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.926 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.918+0000 7f1caccf9700 1 -- 172.21.15.43:0/3889877571 >> v1:172.21.15.43:6789/0 conn(0x7f1ca8158dd0 legacy=0x7f1ca8069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.918+0000 7f1caccf9700 1 -- 172.21.15.43:0/3889877571 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca80b4e40 con 0x7f1ca806a600 2023-06-28T20:08:55.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.918+0000 7f1caccf9700 1 --2- 172.21.15.43:0/3889877571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ca806a600 0x7f1ca80cf160 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80c9720 tx=0x7f1c98009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c565344dd91c642 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:55.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.918+0000 7f1c9ffff700 1 -- 172.21.15.43:0/3889877571 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c98011830 con 0x7f1ca806a600 2023-06-28T20:08:55.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.919+0000 7f1c9ffff700 1 -- 172.21.15.43:0/3889877571 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c98011990 con 0x7f1ca806a600 2023-06-28T20:08:55.927 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.919+0000 7f1c9ffff700 1 -- 172.21.15.43:0/3889877571 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c98011d20 con 0x7f1ca806a600 2023-06-28T20:08:55.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.919+0000 7f1caf75e700 1 -- 172.21.15.43:0/3889877571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ca806a600 msgr2=0x7f1ca80cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.919+0000 7f1caf75e700 1 --2- 172.21.15.43:0/3889877571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ca806a600 0x7f1ca80cf160 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80c9720 tx=0x7f1c98009b50 comp rx=0 tx=0).stop 2023-06-28T20:08:55.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.919+0000 7f1caf75e700 1 -- 172.21.15.43:0/3889877571 shutdown_connections 2023-06-28T20:08:55.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.919+0000 7f1caf75e700 1 --2- 172.21.15.43:0/3889877571 >> v2:172.21.15.178:3300/0 conn(0x7f1ca806f870 0x7f1ca80ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.928 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.919+0000 7f1caf75e700 1 --2- 172.21.15.43:0/3889877571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ca806a600 0x7f1ca80cf160 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.919+0000 7f1caf75e700 1 -- 172.21.15.43:0/3889877571 >> 172.21.15.43:0/3889877571 conn(0x7f1ca80b2ed0 msgr2=0x7f1ca8071460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.919+0000 7f1caf75e700 1 -- 172.21.15.43:0/3889877571 shutdown_connections 2023-06-28T20:08:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.919+0000 7f1caf75e700 1 -- 172.21.15.43:0/3889877571 wait complete. 2023-06-28T20:08:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.920+0000 7f1caf75e700 1 Processor -- start 2023-06-28T20:08:55.930 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.920+0000 7f1caf75e700 1 -- start start 2023-06-28T20:08:55.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.921+0000 7f1caf75e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ca806a600 0x7f1ca80c10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.932 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.921+0000 7f1caf75e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ca806f870 0x7f1ca80c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.921+0000 7f1caf75e700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ca8158dd0 0x7f1ca8153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.921+0000 7f1caf75e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca8159d70 con 0x7f1ca806a600 2023-06-28T20:08:55.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.921+0000 7f1caf75e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca80c8a60 con 0x7f1ca806f870 2023-06-28T20:08:55.933 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.921+0000 7f1caf75e700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca80c9270 con 0x7f1ca8158dd0 2023-06-28T20:08:55.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.921+0000 7f1cad4fa700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ca806a600 0x7f1ca80c10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.921+0000 7f1cad4fa700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ca806a600 0x7f1ca80c10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55916/0 (socket says 172.21.15.43:55916) 2023-06-28T20:08:55.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.921+0000 7f1cad4fa700 1 -- 172.21.15.43:0/622438829 learned_addr learned my addr 172.21.15.43:0/622438829 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:55.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.921+0000 7f1caccf9700 1 --2- 172.21.15.43:0/622438829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ca806f870 0x7f1ca80c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.934 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.921+0000 7f1cadcfb700 1 --2- 172.21.15.43:0/622438829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ca8158dd0 0x7f1ca8153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.921+0000 7f1cad4fa700 1 -- 172.21.15.43:0/622438829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ca8158dd0 msgr2=0x7f1ca8153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.935 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.922+0000 7f1cad4fa700 1 --2- 172.21.15.43:0/622438829 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f1ca8158dd0 0x7f1ca8153ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.922+0000 7f1cad4fa700 1 -- 172.21.15.43:0/622438829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ca806f870 msgr2=0x7f1ca80c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.922+0000 7f1cad4fa700 1 --2- 172.21.15.43:0/622438829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ca806f870 0x7f1ca80c7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.922+0000 7f1cad4fa700 1 -- 172.21.15.43:0/622438829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c980094a0 con 0x7f1ca806a600 2023-06-28T20:08:55.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.922+0000 7f1cad4fa700 1 --2- 172.21.15.43:0/622438829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ca806a600 0x7f1ca80c10f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400c4d0 tx=0x7f1ca400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:55.936 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.922+0000 7f1c9dffb700 1 -- 172.21.15.43:0/622438829 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4005d70 con 0x7f1ca806a600 2023-06-28T20:08:55.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.922+0000 7f1c9dffb700 1 -- 172.21.15.43:0/622438829 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca40070d0 con 0x7f1ca806a600 2023-06-28T20:08:55.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.922+0000 7f1c9dffb700 1 -- 172.21.15.43:0/622438829 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4007460 con 0x7f1ca806a600 2023-06-28T20:08:55.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.922+0000 7f1caf75e700 1 -- 172.21.15.43:0/622438829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca80c82a0 con 0x7f1ca806a600 2023-06-28T20:08:55.937 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.922+0000 7f1caf75e700 1 -- 172.21.15.43:0/622438829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca80c8f90 con 0x7f1ca806a600 2023-06-28T20:08:55.938 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.923+0000 7f1caf75e700 1 -- 172.21.15.43:0/622438829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1ca8071e10 con 0x7f1ca806a600 2023-06-28T20:08:55.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.923+0000 7f1c9dffb700 1 -- 172.21.15.43:0/622438829 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1ca40075c0 con 0x7f1ca806a600 2023-06-28T20:08:55.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.924+0000 7f1c9dffb700 1 --2- 172.21.15.43:0/622438829 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1c8c06ace0 0x7f1c8c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.939 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.924+0000 7f1c9dffb700 1 -- 172.21.15.43:0/622438829 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1ca40886e0 con 0x7f1ca806a600 2023-06-28T20:08:55.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.924+0000 7f1caccf9700 1 --2- 172.21.15.43:0/622438829 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1c8c06ace0 0x7f1c8c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.940 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.924+0000 7f1c9dffb700 1 --2- 172.21.15.43:0/622438829 >> [v2:172.21.15.178:6816/3953458472,v1:172.21.15.178:6817/3953458472] conn(0x7f1c8c071150 0x7f1c8c073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:55.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.924+0000 7f1c9dffb700 1 -- 172.21.15.43:0/622438829 --> [v2:172.21.15.178:6816/3953458472,v1:172.21.15.178:6817/3953458472] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c8c073c30 con 0x7f1c8c071150 2023-06-28T20:08:55.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.924+0000 7f1c9dffb700 1 -- 172.21.15.43:0/622438829 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ca408ad80 con 0x7f1ca806a600 2023-06-28T20:08:55.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.924+0000 7f1cadcfb700 1 --2- 172.21.15.43:0/622438829 >> [v2:172.21.15.178:6816/3953458472,v1:172.21.15.178:6817/3953458472] conn(0x7f1c8c071150 0x7f1c8c073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:55.941 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.924+0000 7f1caccf9700 1 --2- 172.21.15.43:0/622438829 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1c8c06ace0 0x7f1c8c06d190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80c9720 tx=0x7f1c98010000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:55.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.925+0000 7f1cadcfb700 1 --2- 172.21.15.43:0/622438829 >> [v2:172.21.15.178:6816/3953458472,v1:172.21.15.178:6817/3953458472] conn(0x7f1c8c071150 0x7f1c8c073560 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:55.942 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.925+0000 7f1c9dffb700 1 -- 172.21.15.43:0/622438829 <== osd.7 v2:172.21.15.178:6816/3953458472 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1c8c073c30 con 0x7f1c8c071150 2023-06-28T20:08:55.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.940+0000 7f1caf75e700 1 -- 172.21.15.43:0/622438829 --> [v2:172.21.15.178:6816/3953458472,v1:172.21.15.178:6817/3953458472] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1ca80b5120 con 0x7f1c8c071150 2023-06-28T20:08:55.943 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.940+0000 7f1c9dffb700 1 -- 172.21.15.43:0/622438829 <== osd.7 v2:172.21.15.178:6816/3953458472 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f1ca0003a20 con 0x7f1c8c071150 2023-06-28T20:08:55.944 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2023-06-28T20:08:55.945 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-28T20:08:55.945 INFO:teuthology.orchestra.run.smithi043.stdout:} 2023-06-28T20:08:55.946 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.940+0000 7f1caf75e700 1 -- 172.21.15.43:0/622438829 >> [v2:172.21.15.178:6816/3953458472,v1:172.21.15.178:6817/3953458472] conn(0x7f1c8c071150 msgr2=0x7f1c8c073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.940+0000 7f1caf75e700 1 --2- 172.21.15.43:0/622438829 >> [v2:172.21.15.178:6816/3953458472,v1:172.21.15.178:6817/3953458472] conn(0x7f1c8c071150 0x7f1c8c073560 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:55.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.940+0000 7f1caf75e700 1 -- 172.21.15.43:0/622438829 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1c8c06ace0 msgr2=0x7f1c8c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.940+0000 7f1caf75e700 1 --2- 172.21.15.43:0/622438829 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f1c8c06ace0 0x7f1c8c06d190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80c9720 tx=0x7f1c98010000 comp rx=0 tx=0).stop 2023-06-28T20:08:55.947 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.940+0000 7f1caf75e700 1 -- 172.21.15.43:0/622438829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ca806a600 msgr2=0x7f1ca80c10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:55.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.940+0000 7f1caf75e700 1 --2- 172.21.15.43:0/622438829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ca806a600 0x7f1ca80c10f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400c4d0 tx=0x7f1ca400c890 comp rx=0 tx=0).stop 2023-06-28T20:08:55.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.940+0000 7f1cadcfb700 1 -- 172.21.15.43:0/622438829 reap_dead start 2023-06-28T20:08:55.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.940+0000 7f1caf75e700 1 -- 172.21.15.43:0/622438829 shutdown_connections 2023-06-28T20:08:55.948 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.940+0000 7f1caf75e700 1 -- 172.21.15.43:0/622438829 >> 172.21.15.43:0/622438829 conn(0x7f1ca80b2ed0 msgr2=0x7f1ca80abf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:55.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.940+0000 7f1caf75e700 1 -- 172.21.15.43:0/622438829 shutdown_connections 2023-06-28T20:08:55.949 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:55.941+0000 7f1caf75e700 1 -- 172.21.15.43:0/622438829 wait complete. 2023-06-28T20:08:55.955 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-06-28T20:08:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.079+0000 7fbc15309700 1 Processor -- start 2023-06-28T20:08:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.079+0000 7fbc15309700 1 -- start start 2023-06-28T20:08:56.084 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.080+0000 7fbc15309700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbc100c9900 0x7fbc1006ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:56.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.080+0000 7fbc15309700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc1006b420 0x7fbc1006c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:56.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.080+0000 7fbc15309700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc100ce370 con 0x7fbc1006a830 2023-06-28T20:08:56.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.080+0000 7fbc15309700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc100c9d80 con 0x7fbc100c9900 2023-06-28T20:08:56.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.080+0000 7fbc15309700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1006b8a0 con 0x7fbc1006b420 2023-06-28T20:08:56.085 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.080+0000 7fbc067fc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbc100c9900 0x7fbc1006ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:56.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.080+0000 7fbc067fc700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbc100c9900 0x7fbc1006ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55928/0 (socket says 172.21.15.43:55928) 2023-06-28T20:08:56.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.080+0000 7fbc067fc700 1 -- 172.21.15.43:0/2001189011 learned_addr learned my addr 172.21.15.43:0/2001189011 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:56.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.080+0000 7fbc0f7fe700 1 --2- 172.21.15.43:0/2001189011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc1006b420 0x7fbc1006c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:56.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.080+0000 7fbc067fc700 1 -- 172.21.15.43:0/2001189011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc1006b420 msgr2=0x7fbc1006c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:56.086 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.080+0000 7fbc067fc700 1 --2- 172.21.15.43:0/2001189011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc1006b420 0x7fbc1006c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:56.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.080+0000 7fbc067fc700 1 -- 172.21.15.43:0/2001189011 >> v1:172.21.15.178:6789/0 conn(0x7fbc1006a830 legacy=0x7fbc10069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:56.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.080+0000 7fbc067fc700 1 -- 172.21.15.43:0/2001189011 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc100ce610 con 0x7fbc100c9900 2023-06-28T20:08:56.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.081+0000 7fbc067fc700 1 --2- 172.21.15.43:0/2001189011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc100c9900 0x7fbc1006ac00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800bb90 tx=0x7fbbf800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df50dc38ee1a70a server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:56.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.081+0000 7fbc0e7fc700 1 -- 172.21.15.43:0/2001189011 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf800dc20 con 0x7fbc100c9900 2023-06-28T20:08:56.087 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.081+0000 7fbc0e7fc700 1 -- 172.21.15.43:0/2001189011 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf800dd80 con 0x7fbc100c9900 2023-06-28T20:08:56.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.081+0000 7fbc0e7fc700 1 -- 172.21.15.43:0/2001189011 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8017630 con 0x7fbc100c9900 2023-06-28T20:08:56.088 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.081+0000 7fbc15309700 1 -- 172.21.15.43:0/2001189011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc100c9900 msgr2=0x7fbc1006ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:56.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.081+0000 7fbc15309700 1 --2- 172.21.15.43:0/2001189011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc100c9900 0x7fbc1006ac00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800bb90 tx=0x7fbbf800bea0 comp rx=0 tx=0).stop 2023-06-28T20:08:56.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.081+0000 7fbc15309700 1 -- 172.21.15.43:0/2001189011 shutdown_connections 2023-06-28T20:08:56.090 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.081+0000 7fbc15309700 1 --2- 172.21.15.43:0/2001189011 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc1006b420 0x7fbc1006c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:56.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.081+0000 7fbc15309700 1 --2- 172.21.15.43:0/2001189011 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc100c9900 0x7fbc1006ac00 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:56.091 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.081+0000 7fbc15309700 1 -- 172.21.15.43:0/2001189011 >> 172.21.15.43:0/2001189011 conn(0x7fbc100b1a50 msgr2=0x7fbc100cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:56.092 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.081+0000 7fbc15309700 1 -- 172.21.15.43:0/2001189011 shutdown_connections 2023-06-28T20:08:56.093 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.081+0000 7fbc15309700 1 -- 172.21.15.43:0/2001189011 wait complete. 2023-06-28T20:08:56.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.082+0000 7fbc15309700 1 Processor -- start 2023-06-28T20:08:56.096 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.083+0000 7fbc15309700 1 -- start start 2023-06-28T20:08:56.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.083+0000 7fbc15309700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc1006a830 0x7fbc100c6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:56.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.083+0000 7fbc15309700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc1006b420 0x7fbc10146de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:56.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.083+0000 7fbc15309700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc100c9900 0x7fbc101f0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:56.097 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.083+0000 7fbc15309700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc100c8860 con 0x7fbc100c9900 2023-06-28T20:08:56.098 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.083+0000 7fbc15309700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc100c68c0 con 0x7fbc1006b420 2023-06-28T20:08:56.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.083+0000 7fbc15309700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc100c8a70 con 0x7fbc1006a830 2023-06-28T20:08:56.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.084+0000 7fbc0f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc100c9900 0x7fbc101f0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:56.099 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.084+0000 7fbc0f7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc100c9900 0x7fbc101f0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55936/0 (socket says 172.21.15.43:55936) 2023-06-28T20:08:56.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.084+0000 7fbc0f7fe700 1 -- 172.21.15.43:0/1828239559 learned_addr learned my addr 172.21.15.43:0/1828239559 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:56.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.084+0000 7fbc067fc700 1 --2- 172.21.15.43:0/1828239559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc1006b420 0x7fbc10146de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:56.100 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.084+0000 7fbc0effd700 1 --2- 172.21.15.43:0/1828239559 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc1006a830 0x7fbc100c6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:56.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.084+0000 7fbc0f7fe700 1 -- 172.21.15.43:0/1828239559 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc1006a830 msgr2=0x7fbc100c6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:56.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.084+0000 7fbc0f7fe700 1 --2- 172.21.15.43:0/1828239559 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc1006a830 0x7fbc100c6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:56.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.084+0000 7fbc0f7fe700 1 -- 172.21.15.43:0/1828239559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc1006b420 msgr2=0x7fbc10146de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:56.101 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.084+0000 7fbc0f7fe700 1 --2- 172.21.15.43:0/1828239559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc1006b420 0x7fbc10146de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:56.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.084+0000 7fbc0f7fe700 1 -- 172.21.15.43:0/1828239559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf800b7f0 con 0x7fbc100c9900 2023-06-28T20:08:56.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.084+0000 7fbc0f7fe700 1 --2- 172.21.15.43:0/1828239559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc100c9900 0x7fbc101f0d70 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800efc0 tx=0x7fbc0800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:56.102 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.084+0000 7fbc07fff700 1 -- 172.21.15.43:0/1828239559 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc08009ea0 con 0x7fbc100c9900 2023-06-28T20:08:56.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.084+0000 7fbc07fff700 1 -- 172.21.15.43:0/1828239559 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0800f040 con 0x7fbc100c9900 2023-06-28T20:08:56.103 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.084+0000 7fbc15309700 1 -- 172.21.15.43:0/1828239559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc100c8d50 con 0x7fbc100c9900 2023-06-28T20:08:56.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.084+0000 7fbc07fff700 1 -- 172.21.15.43:0/1828239559 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0800ced0 con 0x7fbc100c9900 2023-06-28T20:08:56.104 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.084+0000 7fbc15309700 1 -- 172.21.15.43:0/1828239559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc100c9310 con 0x7fbc100c9900 2023-06-28T20:08:56.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.085+0000 7fbc15309700 1 -- 172.21.15.43:0/1828239559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc100728b0 con 0x7fbc100c9900 2023-06-28T20:08:56.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.086+0000 7fbc07fff700 1 -- 172.21.15.43:0/1828239559 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbc08014480 con 0x7fbc100c9900 2023-06-28T20:08:56.105 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.086+0000 7fbc07fff700 1 --2- 172.21.15.43:0/1828239559 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbbf006ace0 0x7fbbf006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:56.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.086+0000 7fbc07fff700 1 -- 172.21.15.43:0/1828239559 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fbc08089980 con 0x7fbc100c9900 2023-06-28T20:08:56.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.086+0000 7fbc0effd700 1 --2- 172.21.15.43:0/1828239559 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbbf006ace0 0x7fbbf006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:56.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.087+0000 7fbc0effd700 1 --2- 172.21.15.43:0/1828239559 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbbf006ace0 0x7fbbf006d190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbc00007fd0 tx=0x7fbc00009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:56.106 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.090+0000 7fbc07fff700 1 -- 172.21.15.43:0/1828239559 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbc08054d30 con 0x7fbc100c9900 2023-06-28T20:08:56.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.288+0000 7fbc15309700 1 -- 172.21.15.43:0/1828239559 --> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc100cebd0 con 0x7fbbf006ace0 2023-06-28T20:08:56.291 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.288+0000 7fbc07fff700 1 -- 172.21.15.43:0/1828239559 <== mgr.4101 v2:172.21.15.43:6832/406144153 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbc00001cc0 con 0x7fbbf006ace0 2023-06-28T20:08:56.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.290+0000 7fbc15309700 1 -- 172.21.15.43:0/1828239559 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbbf006ace0 msgr2=0x7fbbf006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:56.293 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.290+0000 7fbc15309700 1 --2- 172.21.15.43:0/1828239559 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbbf006ace0 0x7fbbf006d190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbc00007fd0 tx=0x7fbc00009040 comp rx=0 tx=0).stop 2023-06-28T20:08:56.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.290+0000 7fbc15309700 1 -- 172.21.15.43:0/1828239559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc100c9900 msgr2=0x7fbc101f0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:56.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.290+0000 7fbc15309700 1 --2- 172.21.15.43:0/1828239559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc100c9900 0x7fbc101f0d70 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800efc0 tx=0x7fbc0800c3a0 comp rx=0 tx=0).stop 2023-06-28T20:08:56.294 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.290+0000 7fbc15309700 1 -- 172.21.15.43:0/1828239559 shutdown_connections 2023-06-28T20:08:56.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.290+0000 7fbc15309700 1 --2- 172.21.15.43:0/1828239559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc100c9900 0x7fbc101f0d70 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:56.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.290+0000 7fbc15309700 1 --2- 172.21.15.43:0/1828239559 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fbbf006ace0 0x7fbbf006d190 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:56.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.290+0000 7fbc15309700 1 --2- 172.21.15.43:0/1828239559 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fbc1006a830 0x7fbc100c6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:56.295 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.290+0000 7fbc15309700 1 --2- 172.21.15.43:0/1828239559 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc1006b420 0x7fbc10146de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:56.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.290+0000 7fbc15309700 1 -- 172.21.15.43:0/1828239559 >> 172.21.15.43:0/1828239559 conn(0x7fbc100b1a50 msgr2=0x7fbc1006c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:56.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.290+0000 7fbc15309700 1 -- 172.21.15.43:0/1828239559 shutdown_connections 2023-06-28T20:08:56.296 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.290+0000 7fbc15309700 1 -- 172.21.15.43:0/1828239559 wait complete. 2023-06-28T20:08:56.297 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 7 to deep-scrub 2023-06-28T20:08:56.305 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-28T20:08:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.429+0000 7fad7dd4a700 1 Processor -- start 2023-06-28T20:08:56.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.429+0000 7fad7dd4a700 1 -- start start 2023-06-28T20:08:56.446 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.430+0000 7fad7dd4a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad7815c4e0 0x7fad7806a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:56.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.430+0000 7fad7dd4a700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fad7806c8b0 0x7fad7806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:56.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.430+0000 7fad7dd4a700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad7815def0 con 0x7fad7815d680 2023-06-28T20:08:56.447 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.430+0000 7fad7dd4a700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad780cf710 con 0x7fad7806c8b0 2023-06-28T20:08:56.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.430+0000 7fad7dd4a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad780cf150 con 0x7fad7815c4e0 2023-06-28T20:08:56.449 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.430+0000 7fad77fff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fad7815d680 0x7fad780b4930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45128/0 (socket says 172.21.15.43:45128) 2023-06-28T20:08:56.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.430+0000 7fad77fff700 1 -- 172.21.15.43:0/3301830570 learned_addr learned my addr 172.21.15.43:0/3301830570 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:08:56.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.430+0000 7fad777fe700 1 --2- 172.21.15.43:0/3301830570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad7815c4e0 0x7fad7806a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:56.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.430+0000 7fad76ffd700 1 --2- 172.21.15.43:0/3301830570 >> v2:172.21.15.178:3300/0 conn(0x7fad7806c8b0 0x7fad7806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:56.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.430+0000 7fad767fc700 1 -- 172.21.15.43:0/3301830570 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2435518739 0 0) 0x7fad7815def0 con 0x7fad7815d680 2023-06-28T20:08:56.450 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.430+0000 7fad767fc700 1 -- 172.21.15.43:0/3301830570 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad640035b0 con 0x7fad7815d680 2023-06-28T20:08:56.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.430+0000 7fad767fc700 1 -- 172.21.15.43:0/3301830570 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3036251190 0 0) 0x7fad640035b0 con 0x7fad7815d680 2023-06-28T20:08:56.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.430+0000 7fad767fc700 1 -- 172.21.15.43:0/3301830570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad7815c4e0 msgr2=0x7fad7806a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:56.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.430+0000 7fad767fc700 1 --2- 172.21.15.43:0/3301830570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad7815c4e0 0x7fad7806a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:56.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.430+0000 7fad767fc700 1 -- 172.21.15.43:0/3301830570 >> v2:172.21.15.178:3300/0 conn(0x7fad7806c8b0 msgr2=0x7fad7806abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:56.451 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.431+0000 7fad767fc700 1 --2- 172.21.15.43:0/3301830570 >> v2:172.21.15.178:3300/0 conn(0x7fad7806c8b0 0x7fad7806abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:56.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.431+0000 7fad767fc700 1 -- 172.21.15.43:0/3301830570 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad780cf430 con 0x7fad7815d680 2023-06-28T20:08:56.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.431+0000 7fad76ffd700 1 --2- 172.21.15.43:0/3301830570 >> v2:172.21.15.178:3300/0 conn(0x7fad7806c8b0 0x7fad7806abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-28T20:08:56.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.431+0000 7fad767fc700 1 -- 172.21.15.43:0/3301830570 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7fad700019f0 con 0x7fad7815d680 2023-06-28T20:08:56.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.431+0000 7fad767fc700 1 -- 172.21.15.43:0/3301830570 >> v1:172.21.15.43:6789/0 conn(0x7fad7815d680 legacy=0x7fad780b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:56.452 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.431+0000 7fad767fc700 1 --2- 172.21.15.43:0/3301830570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad7815c4e0 0x7fad64003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:56.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.431+0000 7fad767fc700 1 --2- 172.21.15.43:0/3301830570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad64003f80 0x7fad64004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:56.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.431+0000 7fad767fc700 1 --2- 172.21.15.43:0/3301830570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad64004d90 0x7fad64005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:56.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.431+0000 7fad767fc700 1 -- 172.21.15.43:0/3301830570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad780cf710 con 0x7fad64004d90 2023-06-28T20:08:56.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.431+0000 7fad767fc700 1 -- 172.21.15.43:0/3301830570 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad780cf150 con 0x7fad64003f80 2023-06-28T20:08:56.453 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.431+0000 7fad767fc700 1 -- 172.21.15.43:0/3301830570 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6400ba10 con 0x7fad7815c4e0 2023-06-28T20:08:56.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.431+0000 7fad77fff700 1 --2- 172.21.15.43:0/3301830570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad64004d90 0x7fad64005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:56.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.431+0000 7fad777fe700 1 --2- 172.21.15.43:0/3301830570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad7815c4e0 0x7fad64003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:56.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.431+0000 7fad76ffd700 1 --2- 172.21.15.43:0/3301830570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad64003f80 0x7fad64004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:56.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.431+0000 7fad77fff700 1 -- 172.21.15.43:0/3301830570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad7815c4e0 msgr2=0x7fad64003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:56.454 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.431+0000 7fad77fff700 1 --2- 172.21.15.43:0/3301830570 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad7815c4e0 0x7fad64003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:56.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.431+0000 7fad77fff700 1 -- 172.21.15.43:0/3301830570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad64003f80 msgr2=0x7fad64004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:08:56.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.431+0000 7fad77fff700 1 --2- 172.21.15.43:0/3301830570 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad64003f80 0x7fad64004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:56.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.431+0000 7fad77fff700 1 -- 172.21.15.43:0/3301830570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad640035b0 con 0x7fad64004d90 2023-06-28T20:08:56.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.431+0000 7fad77fff700 1 --2- 172.21.15.43:0/3301830570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad64004d90 0x7fad64005200 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fad7000a860 tx=0x7fad64005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a210272d1cda5d3 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:56.455 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.431+0000 7fad767fc700 1 -- 172.21.15.43:0/3301830570 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70012600 con 0x7fad64004d90 2023-06-28T20:08:56.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.431+0000 7fad767fc700 1 -- 172.21.15.43:0/3301830570 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad70012780 con 0x7fad64004d90 2023-06-28T20:08:56.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.431+0000 7fad767fc700 1 -- 172.21.15.43:0/3301830570 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70012b30 con 0x7fad64004d90 2023-06-28T20:08:56.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.432+0000 7fad7dd4a700 1 -- 172.21.15.43:0/3301830570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad64004d90 msgr2=0x7fad64005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:56.456 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.432+0000 7fad7dd4a700 1 --2- 172.21.15.43:0/3301830570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad64004d90 0x7fad64005200 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fad7000a860 tx=0x7fad64005880 comp rx=0 tx=0).stop 2023-06-28T20:08:56.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.432+0000 7fad77fff700 1 -- 172.21.15.43:0/3301830570 reap_dead start 2023-06-28T20:08:56.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.432+0000 7fad7dd4a700 1 -- 172.21.15.43:0/3301830570 shutdown_connections 2023-06-28T20:08:56.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.432+0000 7fad7dd4a700 1 -- 172.21.15.43:0/3301830570 >> 172.21.15.43:0/3301830570 conn(0x7fad780d2eb0 msgr2=0x7fad780d32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:56.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.432+0000 7fad7dd4a700 1 -- 172.21.15.43:0/3301830570 shutdown_connections 2023-06-28T20:08:56.457 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.432+0000 7fad7dd4a700 1 -- 172.21.15.43:0/3301830570 wait complete. 2023-06-28T20:08:56.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.433+0000 7fad7dd4a700 1 Processor -- start 2023-06-28T20:08:56.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.433+0000 7fad7dd4a700 1 -- start start 2023-06-28T20:08:56.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.434+0000 7fad7dd4a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad64004d90 0x7fad781f55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:56.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.434+0000 7fad7dd4a700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad780b28f0 0x7fad780c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:56.458 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.434+0000 7fad7dd4a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad780b2d60 0x7fad780c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:56.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.434+0000 7fad7dd4a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad780c6b60 con 0x7fad64004d90 2023-06-28T20:08:56.459 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.434+0000 7fad7dd4a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad780c6e40 con 0x7fad780b2d60 2023-06-28T20:08:56.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.434+0000 7fad7dd4a700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad780c7120 con 0x7fad780b28f0 2023-06-28T20:08:56.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.434+0000 7fad777fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad64004d90 0x7fad781f55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:56.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.434+0000 7fad777fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad64004d90 0x7fad781f55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55954/0 (socket says 172.21.15.43:55954) 2023-06-28T20:08:56.460 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.434+0000 7fad777fe700 1 -- 172.21.15.43:0/1263955315 learned_addr learned my addr 172.21.15.43:0/1263955315 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:08:56.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.434+0000 7fad77fff700 1 --2- 172.21.15.43:0/1263955315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad780b2d60 0x7fad780c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:56.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.434+0000 7fad76ffd700 1 --2- 172.21.15.43:0/1263955315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad780b28f0 0x7fad780c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:56.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.434+0000 7fad777fe700 1 -- 172.21.15.43:0/1263955315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad780b28f0 msgr2=0x7fad780c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:56.461 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.434+0000 7fad777fe700 1 --2- 172.21.15.43:0/1263955315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad780b28f0 0x7fad780c7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:56.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.434+0000 7fad777fe700 1 -- 172.21.15.43:0/1263955315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad780b2d60 msgr2=0x7fad780c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:56.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.434+0000 7fad777fe700 1 --2- 172.21.15.43:0/1263955315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad780b2d60 0x7fad780c8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:56.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.434+0000 7fad777fe700 1 -- 172.21.15.43:0/1263955315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad780c7400 con 0x7fad64004d90 2023-06-28T20:08:56.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.434+0000 7fad76ffd700 1 --2- 172.21.15.43:0/1263955315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad780b28f0 0x7fad780c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:56.462 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.435+0000 7fad77fff700 1 --2- 172.21.15.43:0/1263955315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad780b2d60 0x7fad780c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:08:56.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.435+0000 7fad777fe700 1 --2- 172.21.15.43:0/1263955315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad64004d90 0x7fad781f55d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fad68011aa0 tx=0x7fad68011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:56.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.435+0000 7fad6ffff700 1 -- 172.21.15.43:0/1263955315 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6800f980 con 0x7fad64004d90 2023-06-28T20:08:56.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.435+0000 7fad6ffff700 1 -- 172.21.15.43:0/1263955315 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6800fae0 con 0x7fad64004d90 2023-06-28T20:08:56.463 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.435+0000 7fad7dd4a700 1 -- 172.21.15.43:0/1263955315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad780c76e0 con 0x7fad64004d90 2023-06-28T20:08:56.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.435+0000 7fad6ffff700 1 -- 172.21.15.43:0/1263955315 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6800fe90 con 0x7fad64004d90 2023-06-28T20:08:56.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.435+0000 7fad7dd4a700 1 -- 172.21.15.43:0/1263955315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad781f5cf0 con 0x7fad64004d90 2023-06-28T20:08:56.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.436+0000 7fad7dd4a700 1 -- 172.21.15.43:0/1263955315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad780728b0 con 0x7fad64004d90 2023-06-28T20:08:56.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.436+0000 7fad6ffff700 1 -- 172.21.15.43:0/1263955315 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fad68014440 con 0x7fad64004d90 2023-06-28T20:08:56.464 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.437+0000 7fad6ffff700 1 --2- 172.21.15.43:0/1263955315 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fad5806ace0 0x7fad5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:08:56.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.437+0000 7fad6ffff700 1 -- 172.21.15.43:0/1263955315 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fad680897a0 con 0x7fad64004d90 2023-06-28T20:08:56.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.437+0000 7fad76ffd700 1 --2- 172.21.15.43:0/1263955315 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fad5806ace0 0x7fad5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:08:56.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.437+0000 7fad76ffd700 1 --2- 172.21.15.43:0/1263955315 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fad5806ace0 0x7fad5806d190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fad780c9970 tx=0x7fad60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:08:56.465 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.441+0000 7fad6ffff700 1 -- 172.21.15.43:0/1263955315 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fad680586a0 con 0x7fad64004d90 2023-06-28T20:08:56.638 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.635+0000 7fad7dd4a700 1 -- 172.21.15.43:0/1263955315 --> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad7815db10 con 0x7fad5806ace0 2023-06-28T20:08:56.641 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.638+0000 7fad6ffff700 1 -- 172.21.15.43:0/1263955315 <== mgr.4101 v2:172.21.15.43:6832/406144153 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72809 (secure 0 0 0) 0x7fad60009e10 con 0x7fad5806ace0 2023-06-28T20:08:56.642 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:08:56.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.640+0000 7fad7dd4a700 1 -- 172.21.15.43:0/1263955315 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fad5806ace0 msgr2=0x7fad5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:56.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.640+0000 7fad7dd4a700 1 --2- 172.21.15.43:0/1263955315 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fad5806ace0 0x7fad5806d190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fad780c9970 tx=0x7fad60008040 comp rx=0 tx=0).stop 2023-06-28T20:08:56.643 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.640+0000 7fad7dd4a700 1 -- 172.21.15.43:0/1263955315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad64004d90 msgr2=0x7fad781f55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:08:56.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.640+0000 7fad7dd4a700 1 --2- 172.21.15.43:0/1263955315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad64004d90 0x7fad781f55d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fad68011aa0 tx=0x7fad68011e60 comp rx=0 tx=0).stop 2023-06-28T20:08:56.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.640+0000 7fad7dd4a700 1 -- 172.21.15.43:0/1263955315 shutdown_connections 2023-06-28T20:08:56.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.640+0000 7fad7dd4a700 1 --2- 172.21.15.43:0/1263955315 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad780b2d60 0x7fad780c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:56.644 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.640+0000 7fad7dd4a700 1 --2- 172.21.15.43:0/1263955315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad64004d90 0x7fad781f55d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:56.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.640+0000 7fad7dd4a700 1 --2- 172.21.15.43:0/1263955315 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fad5806ace0 0x7fad5806d190 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:56.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.640+0000 7fad7dd4a700 1 --2- 172.21.15.43:0/1263955315 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fad780b28f0 0x7fad780c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:08:56.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.640+0000 7fad7dd4a700 1 -- 172.21.15.43:0/1263955315 >> 172.21.15.43:0/1263955315 conn(0x7fad780d2eb0 msgr2=0x7fad7815b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:08:56.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.640+0000 7fad7dd4a700 1 -- 172.21.15.43:0/1263955315 shutdown_connections 2023-06-28T20:08:56.645 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:08:56.640+0000 7fad7dd4a700 1 -- 172.21.15.43:0/1263955315 wait complete. 2023-06-28T20:08:56.646 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-06-28T20:08:56.655 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2023-06-28T20:08:56.010160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220228,"kb_used_data":4612,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754492,"statfs":{"total":773094113280,"available":772868599808,"internally_reserved":0,"allocated":4722688,"data_stored":2649231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002148"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.226142+0000","last_change":"2023-06-28T20:07:15.815425+0000","last_active":"2023-06-28T20:08:42.226142+0000","last_peered":"2023-06-28T20:08:42.226142+0000","last_clean":"2023-06-28T20:08:42.226142+0000","last_became_active":"2023-06-28T20:07:15.815163+0000","last_became_peered":"2023-06-28T20:07:15.815163+0000","last_unstale":"2023-06-28T20:08:42.226142+0000","last_undegraded":"2023-06-28T20:08:42.226142+0000","last_fullsized":"2023-06-28T20:08:42.226142+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:54:07.698762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":49,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.225754+0000","last_change":"2023-06-28T20:07:15.303675+0000","last_active":"2023-06-28T20:08:42.225754+0000","last_peered":"2023-06-28T20:08:42.225754+0000","last_clean":"2023-06-28T20:08:42.225754+0000","last_became_active":"2023-06-28T20:07:15.303350+0000","last_became_peered":"2023-06-28T20:07:15.303350+0000","last_unstale":"2023-06-28T20:08:42.225754+0000","last_undegraded":"2023-06-28T20:08:42.225754+0000","last_fullsized":"2023-06-28T20:08:42.225754+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:51:41.786605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":44,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.225048+0000","last_change":"2023-06-28T20:07:15.303565+0000","last_active":"2023-06-28T20:08:42.225048+0000","last_peered":"2023-06-28T20:08:42.225048+0000","last_clean":"2023-06-28T20:08:42.225048+0000","last_became_active":"2023-06-28T20:07:15.303200+0000","last_became_peered":"2023-06-28T20:07:15.303200+0000","last_unstale":"2023-06-28T20:08:42.225048+0000","last_undegraded":"2023-06-28T20:08:42.225048+0000","last_fullsized":"2023-06-28T20:08:42.225048+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":189,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.094410+0000","last_change":"2023-06-28T20:07:15.304921+0000","last_active":"2023-06-28T20:08:42.094410+0000","last_peered":"2023-06-28T20:08:42.094410+0000","last_clean":"2023-06-28T20:08:42.094410+0000","last_became_active":"2023-06-28T20:07:15.303959+0000","last_became_peered":"2023-06-28T20:07:15.303959+0000","last_unstale":"2023-06-28T20:08:42.094410+0000","last_undegraded":"2023-06-28T20:08:42.094410+0000","last_fullsized":"2023-06-28T20:08:42.094410+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T23:07:09.587950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":41,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.094663+0000","last_change":"2023-06-28T20:07:15.899048+0000","last_active":"2023-06-28T20:08:42.094663+0000","last_peered":"2023-06-28T20:08:42.094663+0000","last_clean":"2023-06-28T20:08:42.094663+0000","last_became_active":"2023-06-28T20:07:15.898766+0000","last_became_peered":"2023-06-28T20:07:15.898766+0000","last_unstale":"2023-06-28T20:08:42.094663+0000","last_undegraded":"2023-06-28T20:08:42.094663+0000","last_fullsized":"2023-06-28T20:08:42.094663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T04:32:41.079504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":39,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.094199+0000","last_change":"2023-06-28T20:07:15.899229+0000","last_active":"2023-06-28T20:08:42.094199+0000","last_peered":"2023-06-28T20:08:42.094199+0000","last_clean":"2023-06-28T20:08:42.094199+0000","last_became_active":"2023-06-28T20:07:15.898708+0000","last_became_peered":"2023-06-28T20:07:15.898708+0000","last_unstale":"2023-06-28T20:08:42.094199+0000","last_undegraded":"2023-06-28T20:08:42.094199+0000","last_fullsized":"2023-06-28T20:08:42.094199+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:11:32.901441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.227358+0000","last_change":"2023-06-28T20:07:15.898966+0000","last_active":"2023-06-28T20:08:42.227358+0000","last_peered":"2023-06-28T20:08:42.227358+0000","last_clean":"2023-06-28T20:08:42.227358+0000","last_became_active":"2023-06-28T20:07:15.898704+0000","last_became_peered":"2023-06-28T20:07:15.898704+0000","last_unstale":"2023-06-28T20:08:42.227358+0000","last_undegraded":"2023-06-28T20:08:42.227358+0000","last_fullsized":"2023-06-28T20:08:42.227358+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:19:46.078630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":53,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.095830+0000","last_change":"2023-06-28T20:07:15.899207+0000","last_active":"2023-06-28T20:08:42.095830+0000","last_peered":"2023-06-28T20:08:42.095830+0000","last_clean":"2023-06-28T20:08:42.095830+0000","last_became_active":"2023-06-28T20:07:15.898908+0000","last_became_peered":"2023-06-28T20:07:15.898908+0000","last_unstale":"2023-06-28T20:08:42.095830+0000","last_undegraded":"2023-06-28T20:08:42.095830+0000","last_fullsized":"2023-06-28T20:08:42.095830+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T20:58:47.410109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":53,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:42.095885+0000","last_change":"2023-06-28T20:07:15.303873+0000","last_active":"2023-06-28T20:08:42.095885+0000","last_peered":"2023-06-28T20:08:42.095885+0000","last_clean":"2023-06-28T20:08:42.095885+0000","last_became_active":"2023-06-28T20:07:15.303457+0000","last_became_peered":"2023-06-28T20:07:15.303457+0000","last_unstale":"2023-06-28T20:08:42.095885+0000","last_undegraded":"2023-06-28T20:08:42.095885+0000","last_fullsized":"2023-06-28T20:08:42.095885+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:07:14.287034+0000","last_clean_scrub_stamp":"2023-06-28T20:07:14.287034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T05:20:41.981562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":1036288,"data_stored":774050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.627},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.53600000000000003}]},{"osd":1,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.64100000000000001}]},{"osd":2,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.61099999999999999}]},{"osd":3,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.65300000000000002}]},{"osd":4,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.42899999999999999}]},{"osd":5,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.69299999999999995}]},{"osd":6,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.5},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.59099999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":183682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.47899999999999998},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":9.0069999999999997,"5min":9.0069999999999997,"15min":9.0069999999999997},"last":0.51100000000000001}]},{"osd":1,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":3.036,"5min":3.036,"15min":3.036},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.501}]},{"osd":2,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.45700000000000002}]},{"osd":3,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":9.0459999999999994,"5min":9.0459999999999994,"15min":9.0459999999999994},"last":0.442},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.43099999999999999}]},{"osd":4,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.33800000000000002}]},{"osd":5,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.40400000000000003}]},{"osd":7,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.374}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":1036288,"data_stored":774050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.47199999999999998}]},{"osd":2,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.495},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.70599999999999996}]},{"osd":3,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.63900000000000001}]},{"osd":4,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.995}]},{"osd":5,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.83199999999999996}]},{"osd":6,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.878},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.80900000000000005}]},{"osd":7,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.96199999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":438272,"data_stored":182721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.39000000000000001}]},{"osd":2,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.442,"5min":0.442,"15min":0.442},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.34499999999999997}]},{"osd":3,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.371},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.44500000000000001}]},{"osd":4,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.72099999999999997}]},{"osd":5,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.73499999999999999}]},{"osd":6,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.70999999999999996}]},{"osd":7,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.64300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":183682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":16.044,"5min":16.044,"15min":16.044},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.318}]},{"osd":1,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":11.028,"5min":11.028,"15min":11.028},"last":0.38800000000000001},{"interface":"front","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":16.077000000000002,"5min":16.077000000000002,"15min":16.077000000000002},"last":0.34300000000000003}]},{"osd":3,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.36399999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.319,"15min":1.319},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":16.004000000000001,"5min":16.004000000000001,"15min":16.004000000000001},"last":0.40500000000000003}]},{"osd":4,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":9.327,"5min":9.327,"15min":9.327},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":10.964,"5min":10.964,"15min":10.964},"last":0.67700000000000005}]},{"osd":5,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":10.923,"5min":10.923,"15min":10.923},"last":0.628},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.61199999999999999}]},{"osd":6,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.57399999999999995},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":9.282,"5min":9.282,"15min":9.282},"last":0.66000000000000003}]},{"osd":7,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":10.996,"5min":10.996,"15min":10.996},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.64600000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":183682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":5.0250000000000004,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":0.35299999999999998}]},{"osd":1,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":5.2309999999999999,"5min":5.2309999999999999,"15min":5.2309999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":5.3250000000000002,"5min":5.3250000000000002,"15min":5.3250000000000002},"last":0.42199999999999999}]},{"osd":2,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":15.927,"5min":15.927,"15min":15.927},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.497}]},{"osd":4,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.621},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.71199999999999997}]},{"osd":5,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":4.9939999999999998,"5min":4.9939999999999998,"15min":4.9939999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.59199999999999997}]},{"osd":6,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":4.9619999999999997,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.65000000000000002}]},{"osd":7,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":4.9210000000000003,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.63},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.72399999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":183682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":9.1809999999999992,"5min":9.1809999999999992,"15min":9.1809999999999992},"last":0.72799999999999998}]},{"osd":1,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.60299999999999998}]},{"osd":2,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":9.0700000000000003,"5min":9.0700000000000003,"15min":9.0700000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.63100000000000001}]},{"osd":3,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.57599999999999996}]},{"osd":5,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.755},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.70099999999999996}]},{"osd":6,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.78200000000000003}]},{"osd":7,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.70999999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":183682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.41799999999999998}]},{"osd":1,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.42999999999999999}]},{"osd":2,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.496},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.50700000000000001}]},{"osd":3,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":5.6139999999999999,"5min":5.6139999999999999,"15min":5.6139999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":5.6500000000000004,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.53300000000000003}]},{"osd":4,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.34599999999999997}]},{"osd":6,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.40200000000000002}]},{"osd":7,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.46200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-28T20:08:56.660 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2023-06-28T20:07:14.287034+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=28, tm_hour=20, tm_min=7, tm_sec=14, tm_wday=2, tm_yday=179, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=28, tm_hour=20, tm_min=8, tm_sec=50, tm_wday=2, tm_yday=179, tm_isdst=0) 2023-06-28T20:08:56.660 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2023-06-28T20:07:14.287034+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=28, tm_hour=20, tm_min=7, tm_sec=14, tm_wday=2, tm_yday=179, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=28, tm_hour=20, tm_min=8, tm_sec=50, tm_wday=2, tm_yday=179, tm_isdst=0) 2023-06-28T20:08:56.661 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2023-06-28T20:07:14.287034+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=28, tm_hour=20, tm_min=7, tm_sec=14, tm_wday=2, tm_yday=179, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=28, tm_hour=20, tm_min=8, tm_sec=50, tm_wday=2, tm_yday=179, tm_isdst=0) 2023-06-28T20:08:56.661 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-28T20:07:14.287034+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=28, tm_hour=20, tm_min=7, tm_sec=14, tm_wday=2, tm_yday=179, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=28, tm_hour=20, tm_min=8, tm_sec=50, tm_wday=2, tm_yday=179, tm_isdst=0) 2023-06-28T20:08:56.661 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2023-06-28T20:07:14.287034+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=28, tm_hour=20, tm_min=7, tm_sec=14, tm_wday=2, tm_yday=179, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=28, tm_hour=20, tm_min=8, tm_sec=50, tm_wday=2, tm_yday=179, tm_isdst=0) 2023-06-28T20:08:56.662 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-06-28T20:07:14.287034+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=28, tm_hour=20, tm_min=7, tm_sec=14, tm_wday=2, tm_yday=179, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=28, tm_hour=20, tm_min=8, tm_sec=50, tm_wday=2, tm_yday=179, tm_isdst=0) 2023-06-28T20:08:56.662 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2023-06-28T20:07:14.287034+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=28, tm_hour=20, tm_min=7, tm_sec=14, tm_wday=2, tm_yday=179, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=28, tm_hour=20, tm_min=8, tm_sec=50, tm_wday=2, tm_yday=179, tm_isdst=0) 2023-06-28T20:08:56.662 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2023-06-28T20:07:14.287034+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=28, tm_hour=20, tm_min=7, tm_sec=14, tm_wday=2, tm_yday=179, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=28, tm_hour=20, tm_min=8, tm_sec=50, tm_wday=2, tm_yday=179, tm_isdst=0) 2023-06-28T20:08:56.663 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2023-06-28T20:07:14.287034+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=28, tm_hour=20, tm_min=7, tm_sec=14, tm_wday=2, tm_yday=179, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=28, tm_hour=20, tm_min=8, tm_sec=50, tm_wday=2, tm_yday=179, tm_isdst=0) 2023-06-28T20:08:56.663 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-28T20:09:16.664 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-28T20:09:16.803 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.800+0000 7f29b3ca4700 1 Processor -- start 2023-06-28T20:09:16.804 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.801+0000 7f29b3ca4700 1 -- start start 2023-06-28T20:09:16.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.801+0000 7f29b3ca4700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7f29ac0d0630 0x7f29ac0d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:09:16.806 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.801+0000 7f29b3ca4700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f29ac0cbd40 0x7f29ac06b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:09:16.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.801+0000 7f29b3ca4700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29ac0ccf10 con 0x7f29ac06b540 2023-06-28T20:09:16.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.801+0000 7f29b3ca4700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29ac06b980 con 0x7f29ac0cbd40 2023-06-28T20:09:16.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.801+0000 7f29b3ca4700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29ac15cd80 con 0x7f29ac0d0630 2023-06-28T20:09:16.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.801+0000 7f29b123f700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f29ac0cbd40 0x7f29ac06b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:09:16.807 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.802+0000 7f29b2241700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f29ac06b540 0x7f29ac0cd680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45146/0 (socket says 172.21.15.43:45146) 2023-06-28T20:09:16.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.802+0000 7f29b2241700 1 -- 172.21.15.43:0/3330808125 learned_addr learned my addr 172.21.15.43:0/3330808125 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:09:16.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.802+0000 7f29b1a40700 1 --2- 172.21.15.43:0/3330808125 >> v2:172.21.15.178:3300/0 conn(0x7f29ac0d0630 0x7f29ac0d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:09:16.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.802+0000 7f29b123f700 1 -- 172.21.15.43:0/3330808125 >> v2:172.21.15.178:3300/0 conn(0x7f29ac0d0630 msgr2=0x7f29ac0d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:09:16.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.802+0000 7f29b123f700 1 --2- 172.21.15.43:0/3330808125 >> v2:172.21.15.178:3300/0 conn(0x7f29ac0d0630 0x7f29ac0d1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:16.808 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.802+0000 7f29b123f700 1 -- 172.21.15.43:0/3330808125 >> v1:172.21.15.43:6789/0 conn(0x7f29ac06b540 legacy=0x7f29ac0cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:09:16.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.802+0000 7f29b0a3e700 1 -- 172.21.15.43:0/3330808125 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2374923808 0 0) 0x7f29ac0ccf10 con 0x7f29ac06b540 2023-06-28T20:09:16.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.802+0000 7f29b123f700 1 -- 172.21.15.43:0/3330808125 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ac0d0a00 con 0x7f29ac0cbd40 2023-06-28T20:09:16.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.802+0000 7f29b1a40700 1 --2- 172.21.15.43:0/3330808125 >> v2:172.21.15.178:3300/0 conn(0x7f29ac0d0630 0x7f29ac0d1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:09:16.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.802+0000 7f29b123f700 1 --2- 172.21.15.43:0/3330808125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29ac0cbd40 0x7f29ac06b000 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009840 tx=0x7f29a0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3efdfb2a2d1b672 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:09:16.809 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.802+0000 7f29b0a3e700 1 -- 172.21.15.43:0/3330808125 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a0011830 con 0x7f29ac0cbd40 2023-06-28T20:09:16.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.802+0000 7f29b0a3e700 1 -- 172.21.15.43:0/3330808125 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a0011990 con 0x7f29ac0cbd40 2023-06-28T20:09:16.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.802+0000 7f29b0a3e700 1 -- 172.21.15.43:0/3330808125 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a0011d20 con 0x7f29ac0cbd40 2023-06-28T20:09:16.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.803+0000 7f29b3ca4700 1 -- 172.21.15.43:0/3330808125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29ac0cbd40 msgr2=0x7f29ac06b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:09:16.810 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.803+0000 7f29b3ca4700 1 --2- 172.21.15.43:0/3330808125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29ac0cbd40 0x7f29ac06b000 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f29a0009840 tx=0x7f29a0009b50 comp rx=0 tx=0).stop 2023-06-28T20:09:16.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.803+0000 7f29b3ca4700 1 -- 172.21.15.43:0/3330808125 shutdown_connections 2023-06-28T20:09:16.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.803+0000 7f29b3ca4700 1 --2- 172.21.15.43:0/3330808125 >> v2:172.21.15.178:3300/0 conn(0x7f29ac0d0630 0x7f29ac0d1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:16.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.803+0000 7f29b3ca4700 1 --2- 172.21.15.43:0/3330808125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29ac0cbd40 0x7f29ac06b000 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:16.811 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.803+0000 7f29b3ca4700 1 -- 172.21.15.43:0/3330808125 >> 172.21.15.43:0/3330808125 conn(0x7f29ac0664b0 msgr2=0x7f29ac09a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:09:16.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.803+0000 7f29b3ca4700 1 -- 172.21.15.43:0/3330808125 shutdown_connections 2023-06-28T20:09:16.812 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.803+0000 7f29b3ca4700 1 -- 172.21.15.43:0/3330808125 wait complete. 2023-06-28T20:09:16.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.805+0000 7f29b3ca4700 1 Processor -- start 2023-06-28T20:09:16.814 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.805+0000 7f29b3ca4700 1 -- start start 2023-06-28T20:09:16.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.806+0000 7f29b3ca4700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29ac06b540 0x7f29ac1f2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:09:16.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.806+0000 7f29b3ca4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29ac0cbd40 0x7f29ac0b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:09:16.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.806+0000 7f29b3ca4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29ac0d0630 0x7f29ac15b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:09:16.815 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.806+0000 7f29b3ca4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac1f31a0 con 0x7f29ac0d0630 2023-06-28T20:09:16.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.806+0000 7f29b3ca4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac1f26d0 con 0x7f29ac0cbd40 2023-06-28T20:09:16.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.806+0000 7f29b3ca4700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac1f2900 con 0x7f29ac06b540 2023-06-28T20:09:16.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.806+0000 7f29b2241700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29ac0d0630 0x7f29ac15b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:09:16.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.806+0000 7f29b2241700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29ac0d0630 0x7f29ac15b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55968/0 (socket says 172.21.15.43:55968) 2023-06-28T20:09:16.816 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.806+0000 7f29b2241700 1 -- 172.21.15.43:0/415846352 learned_addr learned my addr 172.21.15.43:0/415846352 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:09:16.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.806+0000 7f29b1a40700 1 --2- 172.21.15.43:0/415846352 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29ac06b540 0x7f29ac1f2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:09:16.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.806+0000 7f29b123f700 1 --2- 172.21.15.43:0/415846352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29ac0cbd40 0x7f29ac0b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:09:16.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.806+0000 7f29b2241700 1 -- 172.21.15.43:0/415846352 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29ac06b540 msgr2=0x7f29ac1f2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:09:16.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.806+0000 7f29b2241700 1 --2- 172.21.15.43:0/415846352 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29ac06b540 0x7f29ac1f2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:16.817 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.806+0000 7f29b2241700 1 -- 172.21.15.43:0/415846352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29ac0cbd40 msgr2=0x7f29ac0b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:09:16.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.806+0000 7f29b2241700 1 --2- 172.21.15.43:0/415846352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29ac0cbd40 0x7f29ac0b4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:16.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.806+0000 7f29b2241700 1 -- 172.21.15.43:0/415846352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a00094a0 con 0x7f29ac0d0630 2023-06-28T20:09:16.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.806+0000 7f29b123f700 1 --2- 172.21.15.43:0/415846352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29ac0cbd40 0x7f29ac0b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-28T20:09:16.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.806+0000 7f29b2241700 1 --2- 172.21.15.43:0/415846352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29ac0d0630 0x7f29ac15b940 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f29a4002230 tx=0x7f29a400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:09:16.818 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.806+0000 7f299e7fc700 1 -- 172.21.15.43:0/415846352 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a400ac20 con 0x7f29ac0d0630 2023-06-28T20:09:16.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.806+0000 7f299e7fc700 1 -- 172.21.15.43:0/415846352 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a400ada0 con 0x7f29ac0d0630 2023-06-28T20:09:16.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.806+0000 7f299e7fc700 1 -- 172.21.15.43:0/415846352 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a4007310 con 0x7f29ac0d0630 2023-06-28T20:09:16.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.806+0000 7f29b3ca4700 1 -- 172.21.15.43:0/415846352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29ac1f2be0 con 0x7f29ac0d0630 2023-06-28T20:09:16.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.807+0000 7f29b3ca4700 1 -- 172.21.15.43:0/415846352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ac15e7f0 con 0x7f29ac0d0630 2023-06-28T20:09:16.819 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.807+0000 7f29b3ca4700 1 -- 172.21.15.43:0/415846352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29ac15ea10 con 0x7f29ac0d0630 2023-06-28T20:09:16.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.808+0000 7f299e7fc700 1 -- 172.21.15.43:0/415846352 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f29a4007470 con 0x7f29ac0d0630 2023-06-28T20:09:16.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.808+0000 7f299e7fc700 1 --2- 172.21.15.43:0/415846352 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f299006ace0 0x7f299006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:09:16.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.808+0000 7f299e7fc700 1 -- 172.21.15.43:0/415846352 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f29a4088da0 con 0x7f29ac0d0630 2023-06-28T20:09:16.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.808+0000 7f29b1a40700 1 --2- 172.21.15.43:0/415846352 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f299006ace0 0x7f299006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:09:16.820 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.809+0000 7f29b1a40700 1 --2- 172.21.15.43:0/415846352 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f299006ace0 0x7f299006d190 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0caf30 tx=0x7f29a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:09:16.821 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:16.813+0000 7f299e7fc700 1 -- 172.21.15.43:0/415846352 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f29a40540d0 con 0x7f29ac0d0630 2023-06-28T20:09:17.012 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.009+0000 7f29b3ca4700 1 -- 172.21.15.43:0/415846352 --> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f29ac0cc5e0 con 0x7f299006ace0 2023-06-28T20:09:17.015 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.012+0000 7f299e7fc700 1 -- 172.21.15.43:0/415846352 <== mgr.4101 v2:172.21.15.43:6832/406144153 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72765 (secure 0 0 0) 0x7f29a8014460 con 0x7f299006ace0 2023-06-28T20:09:17.016 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:09:17.017 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.014+0000 7f29b3ca4700 1 -- 172.21.15.43:0/415846352 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f299006ace0 msgr2=0x7f299006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:09:17.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.014+0000 7f29b3ca4700 1 --2- 172.21.15.43:0/415846352 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f299006ace0 0x7f299006d190 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f29ac0caf30 tx=0x7f29a8008040 comp rx=0 tx=0).stop 2023-06-28T20:09:17.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.014+0000 7f29b3ca4700 1 -- 172.21.15.43:0/415846352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29ac0d0630 msgr2=0x7f29ac15b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:09:17.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.014+0000 7f29b3ca4700 1 --2- 172.21.15.43:0/415846352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29ac0d0630 0x7f29ac15b940 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f29a4002230 tx=0x7f29a400a060 comp rx=0 tx=0).stop 2023-06-28T20:09:17.018 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.014+0000 7f29b3ca4700 1 -- 172.21.15.43:0/415846352 shutdown_connections 2023-06-28T20:09:17.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.014+0000 7f29b3ca4700 1 --2- 172.21.15.43:0/415846352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29ac0d0630 0x7f29ac15b940 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:17.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.014+0000 7f29b3ca4700 1 --2- 172.21.15.43:0/415846352 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f299006ace0 0x7f299006d190 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:17.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.014+0000 7f29b3ca4700 1 --2- 172.21.15.43:0/415846352 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f29ac06b540 0x7f29ac1f2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:17.019 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.014+0000 7f29b3ca4700 1 --2- 172.21.15.43:0/415846352 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29ac0cbd40 0x7f29ac0b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:17.020 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.014+0000 7f29b3ca4700 1 -- 172.21.15.43:0/415846352 >> 172.21.15.43:0/415846352 conn(0x7f29ac0664b0 msgr2=0x7f29ac0cb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:09:17.024 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.014+0000 7f29b3ca4700 1 -- 172.21.15.43:0/415846352 shutdown_connections 2023-06-28T20:09:17.025 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.014+0000 7f29b3ca4700 1 -- 172.21.15.43:0/415846352 wait complete. 2023-06-28T20:09:17.025 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2023-06-28T20:09:17.029 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":93,"stamp":"2023-06-28T20:09:16.014856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220228,"kb_used_data":4612,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754492,"statfs":{"total":773094113280,"available":772868599808,"internally_reserved":0,"allocated":4722688,"data_stored":2649231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002847"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":52,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:59.361232+0000","last_change":"2023-06-28T20:08:59.361321+0000","last_active":"2023-06-28T20:08:59.361232+0000","last_peered":"2023-06-28T20:08:59.361232+0000","last_clean":"2023-06-28T20:08:59.361232+0000","last_became_active":"2023-06-28T20:07:15.815163+0000","last_became_peered":"2023-06-28T20:07:15.815163+0000","last_unstale":"2023-06-28T20:08:59.361232+0000","last_undegraded":"2023-06-28T20:08:59.361232+0000","last_fullsized":"2023-06-28T20:08:59.361232+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:59.361092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:59.361092+0000","last_clean_scrub_stamp":"2023-06-28T20:08:59.361092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:53:26.860662+0000","scrub_duration":0.0021058779999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":58,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:54.637969+0000","last_change":"2023-06-28T20:08:54.638007+0000","last_active":"2023-06-28T20:08:54.637969+0000","last_peered":"2023-06-28T20:08:54.637969+0000","last_clean":"2023-06-28T20:08:54.637969+0000","last_became_active":"2023-06-28T20:07:15.303350+0000","last_became_peered":"2023-06-28T20:07:15.303350+0000","last_unstale":"2023-06-28T20:08:54.637969+0000","last_undegraded":"2023-06-28T20:08:54.637969+0000","last_fullsized":"2023-06-28T20:08:54.637969+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:54.637872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:54.637872+0000","last_clean_scrub_stamp":"2023-06-28T20:08:54.637872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:17:18.603022+0000","scrub_duration":0.00186695,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":52,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:54.151709+0000","last_change":"2023-06-28T20:08:54.151757+0000","last_active":"2023-06-28T20:08:54.151709+0000","last_peered":"2023-06-28T20:08:54.151709+0000","last_clean":"2023-06-28T20:08:54.151709+0000","last_became_active":"2023-06-28T20:07:15.303200+0000","last_became_peered":"2023-06-28T20:07:15.303200+0000","last_unstale":"2023-06-28T20:08:54.151709+0000","last_undegraded":"2023-06-28T20:08:54.151709+0000","last_fullsized":"2023-06-28T20:08:54.151709+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:54.151594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:54.151594+0000","last_clean_scrub_stamp":"2023-06-28T20:08:54.151594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T21:00:27.274669+0000","scrub_duration":0.0018065220000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"38'119","reported_seq":200,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:57.408106+0000","last_change":"2023-06-28T20:08:57.408187+0000","last_active":"2023-06-28T20:08:57.408106+0000","last_peered":"2023-06-28T20:08:57.408106+0000","last_clean":"2023-06-28T20:08:57.408106+0000","last_became_active":"2023-06-28T20:07:15.303959+0000","last_became_peered":"2023-06-28T20:07:15.303959+0000","last_unstale":"2023-06-28T20:08:57.408106+0000","last_undegraded":"2023-06-28T20:08:57.408106+0000","last_fullsized":"2023-06-28T20:08:57.408106+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"38'119","last_scrub_stamp":"2023-06-28T20:08:57.407990+0000","last_deep_scrub":"38'119","last_deep_scrub_stamp":"2023-06-28T20:08:57.407990+0000","last_clean_scrub_stamp":"2023-06-28T20:08:57.407990+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T02:55:50.819718+0000","scrub_duration":0.004976645,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":50,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:09:03.492262+0000","last_change":"2023-06-28T20:09:03.492322+0000","last_active":"2023-06-28T20:09:03.492262+0000","last_peered":"2023-06-28T20:09:03.492262+0000","last_clean":"2023-06-28T20:09:03.492262+0000","last_became_active":"2023-06-28T20:07:15.898766+0000","last_became_peered":"2023-06-28T20:07:15.898766+0000","last_unstale":"2023-06-28T20:09:03.492262+0000","last_undegraded":"2023-06-28T20:09:03.492262+0000","last_fullsized":"2023-06-28T20:09:03.492262+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:09:03.492144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:09:03.492144+0000","last_clean_scrub_stamp":"2023-06-28T20:09:03.492144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:27:47.592528+0000","scrub_duration":0.0021434430000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":48,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:09:07.536590+0000","last_change":"2023-06-28T20:09:07.536636+0000","last_active":"2023-06-28T20:09:07.536590+0000","last_peered":"2023-06-28T20:09:07.536590+0000","last_clean":"2023-06-28T20:09:07.536590+0000","last_became_active":"2023-06-28T20:07:15.898708+0000","last_became_peered":"2023-06-28T20:07:15.898708+0000","last_unstale":"2023-06-28T20:09:07.536590+0000","last_undegraded":"2023-06-28T20:09:07.536590+0000","last_fullsized":"2023-06-28T20:09:07.536590+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:09:07.536473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:09:07.536473+0000","last_clean_scrub_stamp":"2023-06-28T20:09:07.536473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-06-29T22:48:56.581650+0000","scrub_duration":0.00215745,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":52,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:09:00.100776+0000","last_change":"2023-06-28T20:09:00.100817+0000","last_active":"2023-06-28T20:09:00.100776+0000","last_peered":"2023-06-28T20:09:00.100776+0000","last_clean":"2023-06-28T20:09:00.100776+0000","last_became_active":"2023-06-28T20:07:15.898704+0000","last_became_peered":"2023-06-28T20:07:15.898704+0000","last_unstale":"2023-06-28T20:09:00.100776+0000","last_undegraded":"2023-06-28T20:09:00.100776+0000","last_fullsized":"2023-06-28T20:09:00.100776+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:09:00.100676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:09:00.100676+0000","last_clean_scrub_stamp":"2023-06-28T20:09:00.100676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T06:27:44.201061+0000","scrub_duration":0.001688651,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:58.830751+0000","last_change":"2023-06-28T20:08:58.830800+0000","last_active":"2023-06-28T20:08:58.830751+0000","last_peered":"2023-06-28T20:08:58.830751+0000","last_clean":"2023-06-28T20:08:58.830751+0000","last_became_active":"2023-06-28T20:07:15.898908+0000","last_became_peered":"2023-06-28T20:07:15.898908+0000","last_unstale":"2023-06-28T20:08:58.830751+0000","last_undegraded":"2023-06-28T20:08:58.830751+0000","last_fullsized":"2023-06-28T20:08:58.830751+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:58.830639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:58.830639+0000","last_clean_scrub_stamp":"2023-06-28T20:08:58.830639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T03:16:02.700517+0000","scrub_duration":0.001816029,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":62,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-06-28T20:08:56.876353+0000","last_change":"2023-06-28T20:08:56.876402+0000","last_active":"2023-06-28T20:08:56.876353+0000","last_peered":"2023-06-28T20:08:56.876353+0000","last_clean":"2023-06-28T20:08:56.876353+0000","last_became_active":"2023-06-28T20:07:15.303457+0000","last_became_peered":"2023-06-28T20:07:15.303457+0000","last_unstale":"2023-06-28T20:08:56.876353+0000","last_undegraded":"2023-06-28T20:08:56.876353+0000","last_fullsized":"2023-06-28T20:08:56.876353+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-28T20:08:56.876218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-28T20:08:56.876218+0000","last_clean_scrub_stamp":"2023-06-28T20:08:56.876218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-06-30T01:50:43.955743+0000","scrub_duration":0.001724422,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":1036288,"data_stored":774050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.67500000000000004}]},{"osd":1,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.73699999999999999}]},{"osd":2,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.59999999999999998}]},{"osd":3,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.75}]},{"osd":4,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.51600000000000001}]},{"osd":5,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.621}]},{"osd":6,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.65800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":183682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:09:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.36499999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.748,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.94099999999999995,"15min":1.006},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.70599999999999996,"5min":9.0069999999999997,"15min":9.0069999999999997},"last":0.52600000000000002}]},{"osd":1,"last update":"Wed Jun 28 20:09:12 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.68799999999999994,"15min":0.71199999999999997},"min":{"1min":0.41299999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.73399999999999999,"5min":3.036,"15min":3.036},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.44600000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.72099999999999997,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.45500000000000002}]},{"osd":2,"last update":"Wed Jun 28 20:09:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.65800000000000003,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.377,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76800000000000002,"5min":1.167,"15min":1.167},"last":0.54000000000000004}]},{"osd":3,"last update":"Wed Jun 28 20:09:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":1.024,"15min":1.1000000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.78600000000000003,"5min":9.0459999999999994,"15min":9.0459999999999994},"last":0.505},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63600000000000001,"15min":0.63900000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.75,"5min":1.27,"15min":1.27},"last":0.63200000000000001}]},{"osd":4,"last update":"Wed Jun 28 20:09:12 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.56100000000000005,"15min":0.57199999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.58999999999999997,"15min":0.60199999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.68700000000000006,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.68700000000000006}]},{"osd":5,"last update":"Wed Jun 28 20:09:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.54900000000000004,"15min":0.55200000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.624,"15min":0.63400000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.747,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.60699999999999998}]},{"osd":7,"last update":"Wed Jun 28 20:09:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.65300000000000002,"15min":0.66600000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.75,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.64300000000000002,"15min":0.65900000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.76300000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.621}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":1036288,"data_stored":774050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:09:12 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36899999999999999,"15min":0.36899999999999999},"min":{"1min":0.253,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.59099999999999997,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.49399999999999999,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.48799999999999999}]},{"osd":2,"last update":"Wed Jun 28 20:09:12 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34799999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.52800000000000002,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.43099999999999999,"15min":0.433},"min":{"1min":0.30599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70599999999999996,"5min":0.874,"15min":0.874},"last":0.51600000000000001}]},{"osd":3,"last update":"Wed Jun 28 20:09:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45700000000000002,"15min":0.45800000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.65900000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.438,"5min":0.42999999999999999,"15min":0.42899999999999999},"min":{"1min":0.33700000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"last":0.55000000000000004}]},{"osd":4,"last update":"Wed Jun 28 20:09:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.63700000000000001,"15min":0.64500000000000002},"min":{"1min":0.443,"5min":0.441,"15min":0.441},"max":{"1min":0.81699999999999995,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.65800000000000003,"15min":0.66300000000000003},"min":{"1min":0.44,"5min":0.432,"15min":0.432},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.755}]},{"osd":5,"last update":"Wed Jun 28 20:09:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.64100000000000001,"15min":0.64900000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84599999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64500000000000002,"15min":0.65200000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83199999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.80300000000000005}]},{"osd":6,"last update":"Wed Jun 28 20:09:12 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.64500000000000002,"15min":0.65300000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.88200000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.68600000000000005,"15min":0.69199999999999995},"min":{"1min":0.52000000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85699999999999998,"5min":1.05,"15min":1.05},"last":0.85699999999999998}]},{"osd":7,"last update":"Wed Jun 28 20:09:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.70299999999999996,"15min":0.71199999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94499999999999995,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.83199999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":438272,"data_stored":182721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.313},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.40899999999999997}]},{"osd":2,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.374,"15min":0.374},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.442,"5min":0.442,"15min":0.442},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.45900000000000002}]},{"osd":3,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.441}]},{"osd":4,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.504},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.60399999999999998}]},{"osd":5,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.65600000000000003}]},{"osd":6,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.63400000000000001}]},{"osd":7,"last update":"Wed Jun 28 20:08:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.68200000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":183682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":16.044,"5min":16.044,"15min":16.044},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.57399999999999995}]},{"osd":1,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":11.028,"5min":11.028,"15min":11.028},"last":0.436},{"interface":"front","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":16.077000000000002,"5min":16.077000000000002,"15min":16.077000000000002},"last":0.311}]},{"osd":3,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.49299999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.319,"15min":1.319},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":16.004000000000001,"5min":16.004000000000001,"15min":16.004000000000001},"last":0.51800000000000002}]},{"osd":4,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":9.327,"5min":9.327,"15min":9.327},"last":0.629},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":10.964,"5min":10.964,"15min":10.964},"last":0.70999999999999996}]},{"osd":5,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":10.923,"5min":10.923,"15min":10.923},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.66400000000000003}]},{"osd":6,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.59799999999999998},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":9.282,"5min":9.282,"15min":9.282},"last":0.68000000000000005}]},{"osd":7,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.224,"15min":1.224},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":10.996,"5min":10.996,"15min":10.996},"last":0.745},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.69799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":183682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":5.0250000000000004,"5min":5.0250000000000004,"15min":5.0250000000000004},"last":0.311}]},{"osd":1,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":5.2309999999999999,"5min":5.2309999999999999,"15min":5.2309999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":5.3250000000000002,"5min":5.3250000000000002,"15min":5.3250000000000002},"last":0.41799999999999998}]},{"osd":2,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":15.927,"5min":15.927,"15min":15.927},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.441}]},{"osd":4,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.56299999999999994}]},{"osd":5,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":4.9939999999999998,"5min":4.9939999999999998,"15min":4.9939999999999998},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.63100000000000001}]},{"osd":6,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":4.9619999999999997,"5min":4.9619999999999997,"15min":4.9619999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.462,"5min":1.462,"15min":1.462},"last":0.59199999999999997}]},{"osd":7,"last update":"Wed Jun 28 20:08:12 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":4.9210000000000003,"5min":4.9210000000000003,"15min":4.9210000000000003},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.66600000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":183682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":9.1809999999999992,"5min":9.1809999999999992,"15min":9.1809999999999992},"last":0.69299999999999995}]},{"osd":1,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.80500000000000005}]},{"osd":2,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":9.0700000000000003,"5min":9.0700000000000003,"15min":9.0700000000000003},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.79500000000000004}]},{"osd":3,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.77400000000000002}]},{"osd":5,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.61899999999999999}]},{"osd":6,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.753}]},{"osd":7,"last update":"Wed Jun 28 20:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.67400000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":432,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":442368,"data_stored":183682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.497},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.57499999999999996}]},{"osd":1,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.55100000000000005}]},{"osd":2,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.59399999999999997}]},{"osd":3,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":5.6139999999999999,"5min":5.6139999999999999,"15min":5.6139999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":5.6500000000000004,"5min":5.6500000000000004,"15min":5.6500000000000004},"last":0.61799999999999999}]},{"osd":4,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.36499999999999999}]},{"osd":6,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.41399999999999998}]},{"osd":7,"last update":"Wed Jun 28 20:08:14 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.441}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-28T20:09:17.033 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-06-28T20:09:17.158 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.155+0000 7f453d8e1700 1 Processor -- start 2023-06-28T20:09:17.159 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.155+0000 7f453d8e1700 1 -- start start 2023-06-28T20:09:17.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.156+0000 7f453d8e1700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f45380aec90 0x7f4538160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:09:17.162 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.156+0000 7f453d8e1700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4538160250 con 0x7f453806ccc0 2023-06-28T20:09:17.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.156+0000 7f453d8e1700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4538071410 con 0x7f45380af060 2023-06-28T20:09:17.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.156+0000 7f453d8e1700 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f453815cb30 con 0x7f45380aec90 2023-06-28T20:09:17.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.156+0000 7f4536ffd700 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f45380aec90 0x7f4538160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:09:17.163 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.156+0000 7f45377fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f453806ccc0 0x7f45380b1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45162/0 (socket says 172.21.15.43:45162) 2023-06-28T20:09:17.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.156+0000 7f45377fe700 1 -- 172.21.15.43:0/354261311 learned_addr learned my addr 172.21.15.43:0/354261311 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:09:17.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.157+0000 7f4536ffd700 1 -- 172.21.15.43:0/354261311 >> v1:172.21.15.178:6789/0 conn(0x7f45380af060 legacy=0x7f4538070bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:09:17.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.157+0000 7f4536ffd700 1 -- 172.21.15.43:0/354261311 >> v1:172.21.15.43:6789/0 conn(0x7f453806ccc0 legacy=0x7f45380b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:09:17.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.157+0000 7f4536ffd700 1 -- 172.21.15.43:0/354261311 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45381604f0 con 0x7f45380aec90 2023-06-28T20:09:17.164 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.157+0000 7f4535ffb700 1 -- 172.21.15.43:0/354261311 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1752112765 0 0) 0x7f4538160250 con 0x7f453806ccc0 2023-06-28T20:09:17.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.157+0000 7f4536ffd700 1 --2- 172.21.15.43:0/354261311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45380aec90 0x7f4538160cf0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f452c009840 tx=0x7f452c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f370dfc721304a1 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:09:17.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.157+0000 7f4535ffb700 1 -- 172.21.15.43:0/354261311 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c011830 con 0x7f45380aec90 2023-06-28T20:09:17.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.157+0000 7f4535ffb700 1 -- 172.21.15.43:0/354261311 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452c011990 con 0x7f45380aec90 2023-06-28T20:09:17.165 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.157+0000 7f4535ffb700 1 -- 172.21.15.43:0/354261311 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c011d20 con 0x7f45380aec90 2023-06-28T20:09:17.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.158+0000 7f453d8e1700 1 -- 172.21.15.43:0/354261311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45380aec90 msgr2=0x7f4538160cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:09:17.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.158+0000 7f453d8e1700 1 --2- 172.21.15.43:0/354261311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45380aec90 0x7f4538160cf0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f452c009840 tx=0x7f452c009b50 comp rx=0 tx=0).stop 2023-06-28T20:09:17.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.158+0000 7f453d8e1700 1 -- 172.21.15.43:0/354261311 shutdown_connections 2023-06-28T20:09:17.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.158+0000 7f453d8e1700 1 --2- 172.21.15.43:0/354261311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45380aec90 0x7f4538160cf0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:17.166 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.158+0000 7f453d8e1700 1 -- 172.21.15.43:0/354261311 >> 172.21.15.43:0/354261311 conn(0x7f453806b000 msgr2=0x7f453806b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:09:17.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.158+0000 7f453d8e1700 1 -- 172.21.15.43:0/354261311 shutdown_connections 2023-06-28T20:09:17.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.158+0000 7f453d8e1700 1 -- 172.21.15.43:0/354261311 wait complete. 2023-06-28T20:09:17.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.159+0000 7f453d8e1700 1 Processor -- start 2023-06-28T20:09:17.167 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.159+0000 7f453d8e1700 1 -- start start 2023-06-28T20:09:17.169 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.160+0000 7f453d8e1700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f453806ccc0 0x7f45380c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:09:17.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.160+0000 7f453d8e1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45380aec90 0x7f45380b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:09:17.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.160+0000 7f453d8e1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45380af060 0x7f4538152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:09:17.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.160+0000 7f453d8e1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45380c8670 con 0x7f45380af060 2023-06-28T20:09:17.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.160+0000 7f453d8e1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45380c7a60 con 0x7f45380aec90 2023-06-28T20:09:17.170 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.160+0000 7f453d8e1700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45380c7d40 con 0x7f453806ccc0 2023-06-28T20:09:17.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.160+0000 7f45377fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45380af060 0x7f4538152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:09:17.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.160+0000 7f45377fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45380af060 0x7f4538152b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55982/0 (socket says 172.21.15.43:55982) 2023-06-28T20:09:17.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.160+0000 7f45377fe700 1 -- 172.21.15.43:0/4118763782 learned_addr learned my addr 172.21.15.43:0/4118763782 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:09:17.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.160+0000 7f4536ffd700 1 --2- 172.21.15.43:0/4118763782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f453806ccc0 0x7f45380c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:09:17.171 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.160+0000 7f45367fc700 1 --2- 172.21.15.43:0/4118763782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45380aec90 0x7f45380b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:09:17.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.160+0000 7f45377fe700 1 -- 172.21.15.43:0/4118763782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f453806ccc0 msgr2=0x7f45380c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:09:17.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.160+0000 7f45377fe700 1 --2- 172.21.15.43:0/4118763782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f453806ccc0 0x7f45380c8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:17.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.160+0000 7f45377fe700 1 -- 172.21.15.43:0/4118763782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45380aec90 msgr2=0x7f45380b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:09:17.172 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.160+0000 7f45377fe700 1 --2- 172.21.15.43:0/4118763782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45380aec90 0x7f45380b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:17.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.160+0000 7f45377fe700 1 -- 172.21.15.43:0/4118763782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c0094a0 con 0x7f45380af060 2023-06-28T20:09:17.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.160+0000 7f45377fe700 1 --2- 172.21.15.43:0/4118763782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45380af060 0x7f4538152b70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4530002230 tx=0x7f453000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:09:17.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.161+0000 7f451ffff700 1 -- 172.21.15.43:0/4118763782 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453000aac0 con 0x7f45380af060 2023-06-28T20:09:17.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.161+0000 7f451ffff700 1 -- 172.21.15.43:0/4118763782 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453000ac40 con 0x7f45380af060 2023-06-28T20:09:17.173 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.161+0000 7f453d8e1700 1 -- 172.21.15.43:0/4118763782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45380c8020 con 0x7f45380af060 2023-06-28T20:09:17.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.161+0000 7f451ffff700 1 -- 172.21.15.43:0/4118763782 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45300070d0 con 0x7f45380af060 2023-06-28T20:09:17.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.161+0000 7f453d8e1700 1 -- 172.21.15.43:0/4118763782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45380b56d0 con 0x7f45380af060 2023-06-28T20:09:17.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.162+0000 7f453d8e1700 1 -- 172.21.15.43:0/4118763782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45380c8820 con 0x7f45380af060 2023-06-28T20:09:17.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.162+0000 7f451ffff700 1 -- 172.21.15.43:0/4118763782 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4530007250 con 0x7f45380af060 2023-06-28T20:09:17.174 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.162+0000 7f451ffff700 1 --2- 172.21.15.43:0/4118763782 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f452406ace0 0x7f452406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:09:17.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.163+0000 7f451ffff700 1 -- 172.21.15.43:0/4118763782 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f4530088c40 con 0x7f45380af060 2023-06-28T20:09:17.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.163+0000 7f4536ffd700 1 --2- 172.21.15.43:0/4118763782 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f452406ace0 0x7f452406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:09:17.175 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.163+0000 7f4536ffd700 1 --2- 172.21.15.43:0/4118763782 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f452406ace0 0x7f452406d190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f452c0090b0 tx=0x7f452c01c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:09:17.176 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.167+0000 7f451ffff700 1 -- 172.21.15.43:0/4118763782 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f45380c8820 con 0x7f45380af060 2023-06-28T20:09:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.364+0000 7f453d8e1700 1 -- 172.21.15.43:0/4118763782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f45380c8820 con 0x7f45380af060 2023-06-28T20:09:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.368+0000 7f451ffff700 1 -- 172.21.15.43:0/4118763782 <== mon.0 v2:172.21.15.43:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4530001d00 con 0x7f45380af060 2023-06-28T20:09:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.368+0000 7f451ffff700 1 -- 172.21.15.43:0/4118763782 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f45380c8820 con 0x7f45380af060 2023-06-28T20:09:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.371+0000 7f453d8e1700 1 -- 172.21.15.43:0/4118763782 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f452406ace0 msgr2=0x7f452406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:09:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.371+0000 7f453d8e1700 1 --2- 172.21.15.43:0/4118763782 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f452406ace0 0x7f452406d190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f452c0090b0 tx=0x7f452c01c040 comp rx=0 tx=0).stop 2023-06-28T20:09:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.371+0000 7f453d8e1700 1 -- 172.21.15.43:0/4118763782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45380af060 msgr2=0x7f4538152b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:09:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.371+0000 7f453d8e1700 1 --2- 172.21.15.43:0/4118763782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45380af060 0x7f4538152b70 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4530002230 tx=0x7f453000cf50 comp rx=0 tx=0).stop 2023-06-28T20:09:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.371+0000 7f453d8e1700 1 -- 172.21.15.43:0/4118763782 shutdown_connections 2023-06-28T20:09:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.371+0000 7f453d8e1700 1 --2- 172.21.15.43:0/4118763782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45380af060 0x7f4538152b70 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.371+0000 7f453d8e1700 1 --2- 172.21.15.43:0/4118763782 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7f452406ace0 0x7f452406d190 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.371+0000 7f453d8e1700 1 --2- 172.21.15.43:0/4118763782 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7f453806ccc0 0x7f45380c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:17.375 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.371+0000 7f453d8e1700 1 --2- 172.21.15.43:0/4118763782 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45380aec90 0x7f45380b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.371+0000 7f453d8e1700 1 -- 172.21.15.43:0/4118763782 >> 172.21.15.43:0/4118763782 conn(0x7f453806b000 msgr2=0x7f45380b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:09:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.371+0000 7f453d8e1700 1 -- 172.21.15.43:0/4118763782 shutdown_connections 2023-06-28T20:09:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.371+0000 7f453d8e1700 1 -- 172.21.15.43:0/4118763782 wait complete. 2023-06-28T20:09:17.386 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-06-28T20:09:17.387 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-28T20:09:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.508+0000 7fdaf996f700 1 Processor -- start 2023-06-28T20:09:17.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.508+0000 7fdaf996f700 1 -- start start 2023-06-28T20:09:17.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.509+0000 7fdaf996f700 1 --2- >> v2:172.21.15.178:3300/0 conn(0x7fdaf415c4e0 0x7fdaf406a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:09:17.516 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.509+0000 7fdaf996f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf415def0 con 0x7fdaf406c8b0 2023-06-28T20:09:17.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.509+0000 7fdaf996f700 1 -- --> v1:172.21.15.178:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf40b5750 con 0x7fdaf415d680 2023-06-28T20:09:17.517 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.509+0000 7fdaf996f700 1 -- --> v2:172.21.15.178:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf40cf710 con 0x7fdaf415c4e0 2023-06-28T20:09:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.509+0000 7fdaf27fc700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdaf406c8b0 0x7fdaf40b4930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45172/0 (socket says 172.21.15.43:45172) 2023-06-28T20:09:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.509+0000 7fdaf27fc700 1 -- 172.21.15.43:0/3497036062 learned_addr learned my addr 172.21.15.43:0/3497036062 (peer_addr_for_me v1:172.21.15.43:0/0) 2023-06-28T20:09:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.509+0000 7fdaf1ffb700 1 -- 172.21.15.43:0/3497036062 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3463013754 0 0) 0x7fdaf415def0 con 0x7fdaf406c8b0 2023-06-28T20:09:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.509+0000 7fdaf1ffb700 1 -- 172.21.15.43:0/3497036062 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdadc0035b0 con 0x7fdaf406c8b0 2023-06-28T20:09:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.510+0000 7fdaf2ffd700 1 --2- 172.21.15.43:0/3497036062 >> v2:172.21.15.178:3300/0 conn(0x7fdaf415c4e0 0x7fdaf406a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:09:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.510+0000 7fdaf1ffb700 1 -- 172.21.15.43:0/3497036062 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4146370051 0 0) 0x7fdadc0035b0 con 0x7fdaf406c8b0 2023-06-28T20:09:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.510+0000 7fdaf1ffb700 1 -- 172.21.15.43:0/3497036062 >> v2:172.21.15.178:3300/0 conn(0x7fdaf415c4e0 msgr2=0x7fdaf406a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:09:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.510+0000 7fdaf1ffb700 1 --2- 172.21.15.43:0/3497036062 >> v2:172.21.15.178:3300/0 conn(0x7fdaf415c4e0 0x7fdaf406a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.510+0000 7fdaf1ffb700 1 -- 172.21.15.43:0/3497036062 >> v1:172.21.15.178:6789/0 conn(0x7fdaf415d680 legacy=0x7fdaf40b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:09:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.510+0000 7fdaf1ffb700 1 -- 172.21.15.43:0/3497036062 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf40cf150 con 0x7fdaf406c8b0 2023-06-28T20:09:17.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.510+0000 7fdaf1ffb700 1 -- 172.21.15.43:0/3497036062 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1649962808 0 0) 0x7fdae4001670 con 0x7fdaf406c8b0 2023-06-28T20:09:17.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.510+0000 7fdaf1ffb700 1 -- 172.21.15.43:0/3497036062 >> v1:172.21.15.43:6789/0 conn(0x7fdaf406c8b0 legacy=0x7fdaf40b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:09:17.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.510+0000 7fdaf1ffb700 1 --2- 172.21.15.43:0/3497036062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdadc003a00 0x7fdadc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:09:17.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.510+0000 7fdaf1ffb700 1 --2- 172.21.15.43:0/3497036062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdadc004850 0x7fdadc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:09:17.520 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.510+0000 7fdaf1ffb700 1 --2- 172.21.15.43:0/3497036062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdadc005200 0x7fdadc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:09:17.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.510+0000 7fdaf1ffb700 1 -- 172.21.15.43:0/3497036062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf40cf710 con 0x7fdadc005200 2023-06-28T20:09:17.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.510+0000 7fdaf1ffb700 1 -- 172.21.15.43:0/3497036062 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc00c080 con 0x7fdadc004850 2023-06-28T20:09:17.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.510+0000 7fdaf1ffb700 1 -- 172.21.15.43:0/3497036062 --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdadc00c240 con 0x7fdadc003a00 2023-06-28T20:09:17.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.510+0000 7fdaf2ffd700 1 --2- 172.21.15.43:0/3497036062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdadc003a00 0x7fdadc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:09:17.521 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.510+0000 7fdaf37fe700 1 --2- 172.21.15.43:0/3497036062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdadc005200 0x7fdadc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:09:17.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.510+0000 7fdaf27fc700 1 --2- 172.21.15.43:0/3497036062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdadc004850 0x7fdadc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:09:17.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.510+0000 7fdaf37fe700 1 -- 172.21.15.43:0/3497036062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdadc003a00 msgr2=0x7fdadc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:09:17.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.510+0000 7fdaf37fe700 1 --2- 172.21.15.43:0/3497036062 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdadc003a00 0x7fdadc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:17.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.510+0000 7fdaf37fe700 1 -- 172.21.15.43:0/3497036062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdadc004850 msgr2=0x7fdadc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-28T20:09:17.522 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.510+0000 7fdaf37fe700 1 --2- 172.21.15.43:0/3497036062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdadc004850 0x7fdadc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:17.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.510+0000 7fdaf37fe700 1 -- 172.21.15.43:0/3497036062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdadc0035b0 con 0x7fdadc005200 2023-06-28T20:09:17.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.511+0000 7fdaf37fe700 1 -- 172.21.15.43:0/3497036062 reap_dead start 2023-06-28T20:09:17.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.511+0000 7fdaf37fe700 1 --2- 172.21.15.43:0/3497036062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdadc005200 0x7fdadc00ba10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00c160 tx=0x7fdadc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cbe4a376efc9c26 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:09:17.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.511+0000 7fdaf1ffb700 1 -- 172.21.15.43:0/3497036062 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec001db0 con 0x7fdadc005200 2023-06-28T20:09:17.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.511+0000 7fdaf1ffb700 1 -- 172.21.15.43:0/3497036062 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fdaec00cca0 con 0x7fdadc005200 2023-06-28T20:09:17.523 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.511+0000 7fdaf1ffb700 1 -- 172.21.15.43:0/3497036062 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec003c60 con 0x7fdadc005200 2023-06-28T20:09:17.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.511+0000 7fdaf996f700 1 -- 172.21.15.43:0/3497036062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdadc005200 msgr2=0x7fdadc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:09:17.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.511+0000 7fdaf996f700 1 --2- 172.21.15.43:0/3497036062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdadc005200 0x7fdadc00ba10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00c160 tx=0x7fdadc005880 comp rx=0 tx=0).stop 2023-06-28T20:09:17.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.511+0000 7fdaf996f700 1 -- 172.21.15.43:0/3497036062 shutdown_connections 2023-06-28T20:09:17.524 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.511+0000 7fdaf996f700 1 --2- 172.21.15.43:0/3497036062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdadc005200 0x7fdadc00ba10 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:17.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.511+0000 7fdaf996f700 1 -- 172.21.15.43:0/3497036062 >> 172.21.15.43:0/3497036062 conn(0x7fdaf40d2eb0 msgr2=0x7fdaf40d32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:09:17.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.512+0000 7fdaf996f700 1 -- 172.21.15.43:0/3497036062 shutdown_connections 2023-06-28T20:09:17.525 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.512+0000 7fdaf996f700 1 -- 172.21.15.43:0/3497036062 wait complete. 2023-06-28T20:09:17.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.513+0000 7fdaf996f700 1 Processor -- start 2023-06-28T20:09:17.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.513+0000 7fdaf996f700 1 -- start start 2023-06-28T20:09:17.528 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.514+0000 7fdaf996f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdadc005200 0x7fdaf40caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:09:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.514+0000 7fdaf996f700 1 --2- >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdaf41f5ee0 0x7fdaf4159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:09:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.514+0000 7fdaf996f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdaf40b34f0 0x7fdaf40b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:09:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.514+0000 7fdaf996f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf40b3eb0 con 0x7fdadc005200 2023-06-28T20:09:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.514+0000 7fdaf996f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf4158040 con 0x7fdaf40b34f0 2023-06-28T20:09:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.514+0000 7fdaf996f700 1 -- --> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf41f6350 con 0x7fdaf41f5ee0 2023-06-28T20:09:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.514+0000 7fdaf2ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdadc005200 0x7fdaf40caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:09:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.514+0000 7fdaf2ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdadc005200 0x7fdaf40caf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55998/0 (socket says 172.21.15.43:55998) 2023-06-28T20:09:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.514+0000 7fdaf2ffd700 1 -- 172.21.15.43:0/2757733243 learned_addr learned my addr 172.21.15.43:0/2757733243 (peer_addr_for_me v2:172.21.15.43:0/0) 2023-06-28T20:09:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.515+0000 7fdaf27fc700 1 --2- 172.21.15.43:0/2757733243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdaf41f5ee0 0x7fdaf4159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:09:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.515+0000 7fdaf37fe700 1 --2- 172.21.15.43:0/2757733243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdaf40b34f0 0x7fdaf40b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:09:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.515+0000 7fdaf2ffd700 1 -- 172.21.15.43:0/2757733243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdaf41f5ee0 msgr2=0x7fdaf4159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:09:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.515+0000 7fdaf2ffd700 1 --2- 172.21.15.43:0/2757733243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdaf41f5ee0 0x7fdaf4159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.515+0000 7fdaf2ffd700 1 -- 172.21.15.43:0/2757733243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdaf40b34f0 msgr2=0x7fdaf40b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:09:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.515+0000 7fdaf2ffd700 1 --2- 172.21.15.43:0/2757733243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdaf40b34f0 0x7fdaf40b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.515+0000 7fdaf2ffd700 1 -- 172.21.15.43:0/2757733243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf41f1cd0 con 0x7fdadc005200 2023-06-28T20:09:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.515+0000 7fdaf37fe700 1 --2- 172.21.15.43:0/2757733243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdaf40b34f0 0x7fdaf40b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-28T20:09:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.515+0000 7fdaf2ffd700 1 --2- 172.21.15.43:0/2757733243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdadc005200 0x7fdaf40caf40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdae80119a0 tx=0x7fdae8011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:09:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.515+0000 7fdadbfff700 1 -- 172.21.15.43:0/2757733243 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae800f940 con 0x7fdadc005200 2023-06-28T20:09:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.515+0000 7fdadbfff700 1 -- 172.21.15.43:0/2757733243 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fdae800faa0 con 0x7fdadc005200 2023-06-28T20:09:17.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.515+0000 7fdaf996f700 1 -- 172.21.15.43:0/2757733243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf40cb640 con 0x7fdadc005200 2023-06-28T20:09:17.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.515+0000 7fdadbfff700 1 -- 172.21.15.43:0/2757733243 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae800fe50 con 0x7fdadc005200 2023-06-28T20:09:17.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.515+0000 7fdaf996f700 1 -- 172.21.15.43:0/2757733243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf40cbc00 con 0x7fdadc005200 2023-06-28T20:09:17.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.516+0000 7fdaf996f700 1 -- 172.21.15.43:0/2757733243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf40728b0 con 0x7fdadc005200 2023-06-28T20:09:17.533 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.517+0000 7fdadbfff700 1 -- 172.21.15.43:0/2757733243 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdae801d400 con 0x7fdadc005200 2023-06-28T20:09:17.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.517+0000 7fdadbfff700 1 --2- 172.21.15.43:0/2757733243 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fdae006ace0 0x7fdae006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-28T20:09:17.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.517+0000 7fdadbfff700 1 -- 172.21.15.43:0/2757733243 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fdae8089960 con 0x7fdadc005200 2023-06-28T20:09:17.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.517+0000 7fdaf27fc700 1 --2- 172.21.15.43:0/2757733243 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fdae006ace0 0x7fdae006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-28T20:09:17.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.517+0000 7fdaf27fc700 1 --2- 172.21.15.43:0/2757733243 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fdae006ace0 0x7fdae006d190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdae4001c90 tx=0x7fdae4000ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-28T20:09:17.534 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.521+0000 7fdadbfff700 1 -- 172.21.15.43:0/2757733243 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdae8054c80 con 0x7fdadc005200 2023-06-28T20:09:17.766 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.762+0000 7fdaf996f700 1 -- 172.21.15.43:0/2757733243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdaf40ce8b0 con 0x7fdadc005200 2023-06-28T20:09:17.833 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.763+0000 7fdadbfff700 1 -- 172.21.15.43:0/2757733243 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fdae80582a0 con 0x7fdadc005200 2023-06-28T20:09:17.835 INFO:teuthology.orchestra.run.smithi043.stdout: 2023-06-28T20:09:17.835 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":17,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4733,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6835/1274138939","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6833","nonce":1274138939},{"type":"v1","addr":"172.21.15.178:6835","nonce":1274138939}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":4996,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6834/3882460026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6832","nonce":3882460026},{"type":"v1","addr":"172.21.15.178:6834","nonce":3882460026}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5109,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6836/382468709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":382468709},{"type":"v1","addr":"172.21.15.43:6836","nonce":382468709}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":5460,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6837/2868779363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6835","nonce":2868779363},{"type":"v1","addr":"172.21.15.43:6837","nonce":2868779363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[]} 2023-06-28T20:09:17.838 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.765+0000 7fdaf996f700 1 -- 172.21.15.43:0/2757733243 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fdae006ace0 msgr2=0x7fdae006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:09:17.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.765+0000 7fdaf996f700 1 --2- 172.21.15.43:0/2757733243 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fdae006ace0 0x7fdae006d190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdae4001c90 tx=0x7fdae4000ed0 comp rx=0 tx=0).stop 2023-06-28T20:09:17.839 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.765+0000 7fdaf996f700 1 -- 172.21.15.43:0/2757733243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdadc005200 msgr2=0x7fdaf40caf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-28T20:09:17.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.765+0000 7fdaf996f700 1 --2- 172.21.15.43:0/2757733243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdadc005200 0x7fdaf40caf40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdae80119a0 tx=0x7fdae8011cb0 comp rx=0 tx=0).stop 2023-06-28T20:09:17.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.765+0000 7fdaf996f700 1 -- 172.21.15.43:0/2757733243 shutdown_connections 2023-06-28T20:09:17.840 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.765+0000 7fdaf996f700 1 --2- 172.21.15.43:0/2757733243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdaf40b34f0 0x7fdaf40b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:17.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.765+0000 7fdaf996f700 1 --2- 172.21.15.43:0/2757733243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdadc005200 0x7fdaf40caf40 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:17.841 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.765+0000 7fdaf996f700 1 --2- 172.21.15.43:0/2757733243 >> [v2:172.21.15.43:6832/406144153,v1:172.21.15.43:6833/406144153] conn(0x7fdae006ace0 0x7fdae006d190 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:17.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.765+0000 7fdaf996f700 1 --2- 172.21.15.43:0/2757733243 >> [v2:172.21.15.178:3301/0,v1:172.21.15.178:6790/0] conn(0x7fdaf41f5ee0 0x7fdaf4159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-28T20:09:17.842 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.765+0000 7fdaf996f700 1 -- 172.21.15.43:0/2757733243 >> 172.21.15.43:0/2757733243 conn(0x7fdaf40d2eb0 msgr2=0x7fdaf40cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-28T20:09:17.843 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.765+0000 7fdaf996f700 1 -- 172.21.15.43:0/2757733243 shutdown_connections 2023-06-28T20:09:17.845 INFO:teuthology.orchestra.run.smithi043.stderr:2023-06-28T20:09:17.765+0000 7fdaf996f700 1 -- 172.21.15.43:0/2757733243 wait complete. 2023-06-28T20:09:17.846 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 17 2023-06-28T20:09:17.846 DEBUG:tasks.cephfs.filesystem:already dead... 2023-06-28T20:09:17.847 INFO:teuthology.misc:Shutting down mds daemons... 2023-06-28T20:09:17.848 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-06-28T20:09:17.849 INFO:teuthology.orchestra.run:waiting for 300 2023-06-28T20:09:17.859 INFO:tasks.ceph.mds.a:Stopped 2023-06-28T20:09:17.860 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-06-28T20:09:17.861 INFO:teuthology.orchestra.run:waiting for 300 2023-06-28T20:09:17.871 INFO:tasks.ceph.mds.c:Stopped 2023-06-28T20:09:17.873 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-06-28T20:09:17.874 INFO:teuthology.orchestra.run:waiting for 300 2023-06-28T20:09:17.888 INFO:tasks.ceph.mds.b:Stopped 2023-06-28T20:09:17.890 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-06-28T20:09:17.891 INFO:teuthology.orchestra.run:waiting for 300 2023-06-28T20:09:17.941 INFO:tasks.ceph.mds.d:Stopped 2023-06-28T20:09:17.942 INFO:teuthology.misc:Shutting down osd daemons... 2023-06-28T20:09:17.945 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-06-28T20:09:17.945 INFO:teuthology.orchestra.run:waiting for 300 2023-06-28T20:09:17.982 INFO:tasks.ceph.osd.0:Stopped 2023-06-28T20:09:17.983 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-06-28T20:09:17.984 INFO:teuthology.orchestra.run:waiting for 300 2023-06-28T20:09:18.057 INFO:tasks.ceph.osd.1:Stopped 2023-06-28T20:09:18.058 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-06-28T20:09:18.059 INFO:teuthology.orchestra.run:waiting for 300 2023-06-28T20:09:18.136 INFO:tasks.ceph.osd.2:Stopped 2023-06-28T20:09:18.137 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-06-28T20:09:18.137 INFO:teuthology.orchestra.run:waiting for 300 2023-06-28T20:09:18.209 INFO:tasks.ceph.osd.3:Stopped 2023-06-28T20:09:18.210 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-06-28T20:09:18.210 INFO:teuthology.orchestra.run:waiting for 300 2023-06-28T20:09:18.245 INFO:tasks.ceph.osd.4:Stopped 2023-06-28T20:09:18.245 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-06-28T20:09:18.246 INFO:teuthology.orchestra.run:waiting for 300 2023-06-28T20:09:18.314 INFO:tasks.ceph.osd.5:Stopped 2023-06-28T20:09:18.315 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-06-28T20:09:18.315 INFO:teuthology.orchestra.run:waiting for 300 2023-06-28T20:09:18.394 INFO:tasks.ceph.osd.6:Stopped 2023-06-28T20:09:18.395 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-06-28T20:09:18.395 INFO:teuthology.orchestra.run:waiting for 300 2023-06-28T20:09:18.470 INFO:tasks.ceph.osd.7:Stopped 2023-06-28T20:09:18.470 INFO:teuthology.misc:Shutting down mgr daemons... 2023-06-28T20:09:18.471 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-06-28T20:09:18.471 INFO:teuthology.orchestra.run:waiting for 300 2023-06-28T20:09:18.496 INFO:tasks.ceph.mgr.y:Stopped 2023-06-28T20:09:18.496 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-06-28T20:09:18.497 INFO:teuthology.orchestra.run:waiting for 300 2023-06-28T20:09:18.536 INFO:tasks.ceph.mgr.x:Stopped 2023-06-28T20:09:18.536 INFO:teuthology.misc:Shutting down mon daemons... 2023-06-28T20:09:18.537 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-06-28T20:09:18.537 INFO:teuthology.orchestra.run:waiting for 300 2023-06-28T20:09:18.548 INFO:tasks.ceph.mon.a:Stopped 2023-06-28T20:09:18.549 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-06-28T20:09:18.549 INFO:teuthology.orchestra.run:waiting for 300 2023-06-28T20:09:18.587 INFO:tasks.ceph.mon.b:Stopped 2023-06-28T20:09:18.588 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-06-28T20:09:18.589 INFO:teuthology.orchestra.run:waiting for 300 2023-06-28T20:09:18.640 INFO:tasks.ceph.mon.c:Stopped 2023-06-28T20:09:18.641 INFO:tasks.ceph:Checking cluster log for badness... 2023-06-28T20:09:18.641 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-06-28T20:09:18.675 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi043.front.sepia.ceph.com 2023-06-28T20:09:18.676 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-06-28T20:09:18.940 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi043.front.sepia.ceph.com 2023-06-28T20:09:18.941 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-06-28T20:09:19.148 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi043.front.sepia.ceph.com 2023-06-28T20:09:19.149 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-06-28T20:09:19.235 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi043.front.sepia.ceph.com 2023-06-28T20:09:19.236 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-06-28T20:09:19.337 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi178.front.sepia.ceph.com 2023-06-28T20:09:19.338 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-06-28T20:09:19.510 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi178.front.sepia.ceph.com 2023-06-28T20:09:19.511 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-06-28T20:09:19.648 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi178.front.sepia.ceph.com 2023-06-28T20:09:19.648 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-06-28T20:09:19.725 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi178.front.sepia.ceph.com 2023-06-28T20:09:19.726 DEBUG:teuthology.orchestra.run.smithi178:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-06-28T20:09:19.800 INFO:tasks.ceph:Archiving mon data... 2023-06-28T20:09:19.801 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2023-06-28_10:04:29-fs-wip-vshankar-testing-20230628.043830-testing-default-smithi/7319991/data/mon.a.tgz 2023-06-28T20:09:19.803 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2023-06-28T20:09:19.822 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.q082uwxf35 2023-06-28T20:09:19.823 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.q082uwxf35 2023-06-28T20:09:19.954 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.q082uwxf35 2023-06-28T20:09:20.051 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.q082uwxf35 is 283KB 2023-06-28T20:09:20.162 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.q082uwxf35 2023-06-28T20:09:20.189 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2023-06-28_10:04:29-fs-wip-vshankar-testing-20230628.043830-testing-default-smithi/7319991/data/mon.b.tgz 2023-06-28T20:09:20.189 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2023-06-28T20:09:20.209 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.g22mtTTJ9Q 2023-06-28T20:09:20.211 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.g22mtTTJ9Q 2023-06-28T20:09:20.350 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.g22mtTTJ9Q 2023-06-28T20:09:20.450 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.g22mtTTJ9Q is 380KB 2023-06-28T20:09:20.525 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.g22mtTTJ9Q 2023-06-28T20:09:20.550 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2023-06-28_10:04:29-fs-wip-vshankar-testing-20230628.043830-testing-default-smithi/7319991/data/mon.c.tgz 2023-06-28T20:09:20.551 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2023-06-28T20:09:20.616 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.TXO1DuMQ8d 2023-06-28T20:09:20.617 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.TXO1DuMQ8d 2023-06-28T20:09:20.758 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.TXO1DuMQ8d 2023-06-28T20:09:20.853 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.TXO1DuMQ8d is 380KB 2023-06-28T20:09:20.898 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.TXO1DuMQ8d 2023-06-28T20:09:20.916 INFO:tasks.ceph:Cleaning ceph cluster... 2023-06-28T20:09:20.917 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-06-28T20:09:20.920 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-06-28T20:09:20.987 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-06-28T20:09:20.987 DEBUG:teuthology.orchestra.run.smithi043:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-06-28T20:09:20.997 DEBUG:teuthology.orchestra.run.smithi178:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-06-28T20:09:21.030 INFO:teuthology.orchestra.run.smithi043.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-06-28T20:09:21.063 INFO:teuthology.orchestra.run.smithi178.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-06-28T20:09:21.068 INFO:tasks.ceph:Archiving crash dumps... 2023-06-28T20:09:21.070 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-06-28_10:04:29-fs-wip-vshankar-testing-20230628.043830-testing-default-smithi/7319991/remote/smithi043/crash 2023-06-28T20:09:21.074 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-06-28T20:09:21.110 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-06-28_10:04:29-fs-wip-vshankar-testing-20230628.043830-testing-default-smithi/7319991/remote/smithi178/crash 2023-06-28T20:09:21.113 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-06-28T20:09:21.144 INFO:tasks.ceph:Shutting down logrotate 2023-06-28T20:09:21.145 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-28T20:09:21.161 DEBUG:teuthology.orchestra.run.smithi178:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-28T20:09:21.284 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-06-28T20:09:21.317 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-06-28T20:09:21.343 INFO:tasks.ceph:Compressing logs... 2023-06-28T20:09:21.344 DEBUG:teuthology.orchestra.run.smithi043:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-06-28T20:09:21.363 DEBUG:teuthology.orchestra.run.smithi178:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-06-28T20:09:21.391 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139466.log 2023-06-28T20:09:21.392 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143074.log 2023-06-28T20:09:21.393 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139328.log 2023-06-28T20:09:21.393 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.143074.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.139466.log.gz 2023-06-28T20:09:21.394 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145070.log 2023-06-28T20:09:21.394 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143863.log 2023-06-28T20:09:21.394 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.143074.log.gz 2023-06-28T20:09:21.394 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.139328.log.gz 2023-06-28T20:09:21.394 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147101.log 2023-06-28T20:09:21.395 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.143863.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.145070.log.gz 2023-06-28T20:09:21.395 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145749.log 2023-06-28T20:09:21.395 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136993.log 2023-06-28T20:09:21.396 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148842.log 2023-06-28T20:09:21.396 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147101.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.143863.log.gz 2023-06-28T20:09:21.401 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145749.log: /var/log/ceph/ceph-client.admin.136993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144341.log 2023-06-28T20:09:21.401 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.147101.log.gz 2023-06-28T20:09:21.402 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.148842.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.136993.log.gz 2023-06-28T20:09:21.402 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145116.log 2023-06-28T20:09:21.402 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.148842.log.gz 2023-06-28T20:09:21.402 INFO:teuthology.orchestra.run.smithi043.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.145749.log.gz 2023-06-28T20:09:21.403 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.144341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138795.log 2023-06-28T20:09:21.403 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137626.log 2023-06-28T20:09:21.404 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.144341.log.gz 2023-06-28T20:09:21.404 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.145116.log.gz 2023-06-28T20:09:21.404 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146806.log 2023-06-28T20:09:21.405 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.138795.log.gz 2023-06-28T20:09:21.405 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147193.log 2023-06-28T20:09:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146806.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137626.log.gz 2023-06-28T20:09:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.141089.log 2023-06-28T20:09:21.406 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147193.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.146806.log.gz 2023-06-28T20:09:21.407 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-06-28T20:09:21.407 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.147193.log.gz 2023-06-28T20:09:21.408 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.2.141089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139512.log 2023-06-28T20:09:21.408 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.141089.log.gz 2023-06-28T20:09:21.408 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137559.log 2023-06-28T20:09:21.409 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138427.log 2023-06-28T20:09:21.409 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.139512.log.gz 2023-06-28T20:09:21.410 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147009.log 2023-06-28T20:09:21.410 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.137559.log.gz 2023-06-28T20:09:21.410 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136765.log 2023-06-28T20:09:21.411 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.138427.log.gz 2023-06-28T20:09:21.411 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141541.log 2023-06-28T20:09:21.411 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.136765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143223.log 2023-06-28T20:09:21.411 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.147009.log.gz 2023-06-28T20:09:21.412 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.136765.log.gz 2023-06-28T20:09:21.412 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141541.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-06-28T20:09:21.412 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.143223.log: gzip 88.5%-5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.141541.log.gz/var/log/ceph/ceph-client.admin.143028.log 2023-06-28T20:09:21.413 INFO:teuthology.orchestra.run.smithi043.stderr: 2023-06-28T20:09:21.413 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.143223.log.gz 2023-06-28T20:09:21.413 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148658.log 2023-06-28T20:09:21.413 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.143028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146760.log 2023-06-28T20:09:21.413 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.148658.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.143028.log.gz 2023-06-28T20:09:21.414 INFO:teuthology.orchestra.run.smithi043.stderr: 84.7%gzip -5 -- replaced with /var/log/ceph/ceph.log.gz--verbose -- 2023-06-28T20:09:21.414 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.136674.log 2023-06-28T20:09:21.414 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- 87.9%/var/log/ceph/ceph-client.admin.147285.log 2023-06-28T20:09:21.414 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.148658.log.gz 2023-06-28T20:09:21.424 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-06-28T20:09:21.425 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131358.log 2023-06-28T20:09:21.425 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-06-28T20:09:21.425 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-client.admin.131358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131451.log 2023-06-28T20:09:21.425 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.131358.log.gz 2023-06-28T20:09:21.426 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131264.log 2023-06-28T20:09:21.426 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.137447.log 2023-06-28T20:09:21.426 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.131451.log: /var/log/ceph/ceph-client.admin.131264.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-06-28T20:09:21.426 INFO:teuthology.orchestra.run.smithi178.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.131451.log.gz 2023-06-28T20:09:21.426 INFO:teuthology.orchestra.run.smithi178.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.131264.log.gz 2023-06-28T20:09:21.427 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.1.137447.log: 91.4% -- replaced with /var/log/ceph/ceph.log.gz 2023-06-28T20:09:21.427 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.137455.log 2023-06-28T20:09:21.427 INFO:teuthology.orchestra.run.smithi178.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.137447.log.gz 2023-06-28T20:09:21.427 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131311.log 2023-06-28T20:09:21.427 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.1.137455.log: gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.1.137455.log.gz 2023-06-28T20:09:21.428 INFO:teuthology.orchestra.run.smithi178.stderr:--verbose -- /var/log/ceph/ceph-mds.d.log 2023-06-28T20:09:21.428 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.131311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.138837.log 2023-06-28T20:09:21.428 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mds.d.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.131311.log.gz 2023-06-28T20:09:21.428 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-06-28T20:09:21.428 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-06-28T20:09:21.429 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.1.138837.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.138837.log.gz 2023-06-28T20:09:21.429 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-06-28T20:09:21.429 INFO:teuthology.orchestra.run.smithi178.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-06-28T20:09:21.429 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.138062.log 2023-06-28T20:09:21.429 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-06-28T20:09:21.430 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-client.3.138062.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.138062.log.gz 2023-06-28T20:09:21.430 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mon.c.log: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-06-28T20:09:21.430 INFO:teuthology.orchestra.run.smithi178.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-06-28T20:09:21.430 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.138833.log 2023-06-28T20:09:21.430 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131404.log 2023-06-28T20:09:21.431 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.1.138833.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.138833.log.gz 2023-06-28T20:09:21.431 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.131404.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.131404.log.gz 2023-06-28T20:09:21.431 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-06-28T20:09:21.431 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131203.log 2023-06-28T20:09:21.432 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131497.log 2023-06-28T20:09:21.432 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.131203.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.131203.log.gz 2023-06-28T20:09:21.432 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.131497.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.131497.log.gz 2023-06-28T20:09:21.432 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131542.log 2023-06-28T20:09:21.433 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-06-28T20:09:21.433 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.admin.131542.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.131542.log.gz 2023-06-28T20:09:21.435 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-06-28T20:09:21.436 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.138066.log 2023-06-28T20:09:21.436 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-client.3.138066.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.138066.log.gz 2023-06-28T20:09:21.436 INFO:teuthology.orchestra.run.smithi178.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-06-28T20:09:21.449 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146760.log: /var/log/ceph/ceph-client.admin.136674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142378.log 2023-06-28T20:09:21.449 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146760.log.gz 2023-06-28T20:09:21.449 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147285.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.136674.log.gz 2023-06-28T20:09:21.450 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140654.log 2023-06-28T20:09:21.450 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.147285.log.gz 2023-06-28T20:09:21.450 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138055.log 2023-06-28T20:09:21.450 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142378.log: /var/log/ceph/ceph-client.admin.140654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147331.log 2023-06-28T20:09:21.451 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.142378.log.gz 2023-06-28T20:09:21.451 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.140654.log.gz 2023-06-28T20:09:21.451 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131370.log 2023-06-28T20:09:21.451 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147331.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.138055.log.gz 2023-06-28T20:09:21.451 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148750.log 2023-06-28T20:09:21.452 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.147331.log.gz 2023-06-28T20:09:21.452 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.131370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147939.log 2023-06-28T20:09:21.452 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.131370.log.gz 2023-06-28T20:09:21.452 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.148750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137223.log 2023-06-28T20:09:21.453 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0%/var/log/ceph/ceph-client.admin.147939.log: -- replaced with /var/log/ceph/ceph-client.admin.148750.log.gz 2023-06-28T20:09:21.453 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138423.log 2023-06-28T20:09:21.453 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.147939.log.gz 2023-06-28T20:09:21.453 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.140432.log 2023-06-28T20:09:21.454 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138423.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.137223.log.gz 2023-06-28T20:09:21.454 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144886.log 2023-06-28T20:09:21.454 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1%/var/log/ceph/ceph-client.0.140432.log: -- replaced with /var/log/ceph/ceph-client.admin.138423.log.gz 2023-06-28T20:09:21.454 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142194.log 2023-06-28T20:09:21.455 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.140432.log.gz 2023-06-28T20:09:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.144886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148704.log 2023-06-28T20:09:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142194.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.144886.log.gz 2023-06-28T20:09:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138796.log 2023-06-28T20:09:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.148704.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.142194.log.gz 2023-06-28T20:09:21.456 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137718.log 2023-06-28T20:09:21.456 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.148704.log.gz 2023-06-28T20:09:21.456 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138796.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-06-28T20:09:21.460 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.138796.log.gz 2023-06-28T20:09:21.460 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.140424.log 2023-06-28T20:09:21.460 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0%/var/log/ceph/ceph-osd.0.log: -- replaced with /var/log/ceph/ceph-client.admin.137718.log.gz 2023-06-28T20:09:21.461 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148591.log 2023-06-28T20:09:21.461 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.140424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144524.log 2023-06-28T20:09:21.461 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.140424.log.gz 2023-06-28T20:09:21.461 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.148591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139098.log 2023-06-28T20:09:21.462 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.148591.log.gz 2023-06-28T20:09:21.462 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.144524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143269.log 2023-06-28T20:09:21.462 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.144524.log.gz 2023-06-28T20:09:21.462 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147626.log 2023-06-28T20:09:21.462 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.139098.log.gz 2023-06-28T20:09:21.463 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.143269.log: -- /var/log/ceph/ceph-client.admin.144231.log 2023-06-28T20:09:21.463 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148545.log 2023-06-28T20:09:21.463 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147626.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.143269.log.gz 2023-06-28T20:09:21.463 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.144231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144093.log 2023-06-28T20:09:21.464 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.147626.log.gz 2023-06-28T20:09:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.148545.log: 88.2%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.144231.log.gz--verbose -- 2023-06-28T20:09:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.144185.log 2023-06-28T20:09:21.464 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.148545.log.gz 2023-06-28T20:09:21.465 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.144093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148796.log 2023-06-28T20:09:21.465 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141357.log 2023-06-28T20:09:21.465 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.144185.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.144093.log.gz 2023-06-28T20:09:21.465 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.148796.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-06-28T20:09:21.465 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.144185.log.gz 2023-06-28T20:09:21.466 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141357.log: 87.9%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.148796.log.gz--verbose 2023-06-28T20:09:21.466 INFO:teuthology.orchestra.run.smithi043.stderr:-- /var/log/ceph/ceph-client.admin.137131.log 2023-06-28T20:09:21.466 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.141357.log.gz 2023-06-28T20:09:21.466 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142982.log 2023-06-28T20:09:21.466 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141726.log 2023-06-28T20:09:21.467 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145611.log 2023-06-28T20:09:21.467 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137131.log.gz 2023-06-28T20:09:21.467 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6%/var/log/ceph/ceph-client.admin.141726.log: -- replaced with /var/log/ceph/ceph-client.admin.142982.log.gz 2023-06-28T20:09:21.467 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.141726.log.gz 2023-06-28T20:09:21.467 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145611.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.145611.log.gz 2023-06-28T20:09:21.468 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145266.log 2023-06-28T20:09:21.468 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145565.log 2023-06-28T20:09:21.468 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142148.log 2023-06-28T20:09:21.468 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145565.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.145266.log.gz 2023-06-28T20:09:21.468 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.145565.log.gz 2023-06-28T20:09:21.469 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142148.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.142148.log.gz 2023-06-28T20:09:21.469 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137856.log 2023-06-28T20:09:21.469 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142240.log 2023-06-28T20:09:21.469 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139742.log 2023-06-28T20:09:21.470 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142240.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137856.log.gz 2023-06-28T20:09:21.470 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.142240.log.gz 2023-06-28T20:09:21.470 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139742.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.139742.log.gz 2023-06-28T20:09:21.470 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139282.log 2023-06-28T20:09:21.470 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141956.log 2023-06-28T20:09:21.471 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139788.log 2023-06-28T20:09:21.471 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2%/var/log/ceph/ceph-client.admin.141956.log: -- replaced with /var/log/ceph/ceph-client.admin.139282.log.gz 2023-06-28T20:09:21.471 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.141956.log.gz 2023-06-28T20:09:21.471 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142048.log 2023-06-28T20:09:21.472 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148979.log 2023-06-28T20:09:21.472 INFO:teuthology.orchestra.run.smithi043.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.139788.log.gz 2023-06-28T20:09:21.472 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142286.log 2023-06-28T20:09:21.472 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.148979.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.142048.log.gz 2023-06-28T20:09:21.472 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.148979.log.gz 2023-06-28T20:09:21.473 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142286.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.142286.log.gz 2023-06-28T20:09:21.473 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142936.log 2023-06-28T20:09:21.473 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-06-28T20:09:21.473 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145657.log 2023-06-28T20:09:21.474 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.3.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.142936.log.gz 2023-06-28T20:09:21.474 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138638.log 2023-06-28T20:09:21.474 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138797.log 2023-06-28T20:09:21.475 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.145657.log.gz 2023-06-28T20:09:21.475 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138638.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.138638.log.gz 2023-06-28T20:09:21.475 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149025.log 2023-06-28T20:09:21.475 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138797.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.138797.log.gz 2023-06-28T20:09:21.475 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.149025.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.149025.log.gz 2023-06-28T20:09:21.476 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141818.log 2023-06-28T20:09:21.476 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141495.log 2023-06-28T20:09:21.476 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141818.log: /var/log/ceph/ceph-client.admin.141495.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.141818.log.gz 2023-06-28T20:09:21.476 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.141495.log.gz 2023-06-28T20:09:21.476 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147672.log 2023-06-28T20:09:21.477 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138429.log 2023-06-28T20:09:21.477 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147672.log: /var/log/ceph/ceph-client.admin.138429.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.147672.log.gz 2023-06-28T20:09:21.477 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.138429.log.gz 2023-06-28T20:09:21.477 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136563.log 2023-06-28T20:09:21.477 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138063.log 2023-06-28T20:09:21.478 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.136563.log: /var/log/ceph/ceph-client.admin.138063.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.136563.log.gz 2023-06-28T20:09:21.478 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.138063.log.gz 2023-06-28T20:09:21.478 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143361.log 2023-06-28T20:09:21.478 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148407.log 2023-06-28T20:09:21.478 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.143361.log: /var/log/ceph/ceph-client.admin.148407.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.143361.log.gz 2023-06-28T20:09:21.479 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139696.log 2023-06-28T20:09:21.479 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.148407.log.gz 2023-06-28T20:09:21.479 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137039.log 2023-06-28T20:09:21.479 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139696.log: /var/log/ceph/ceph-client.admin.137039.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.139696.log.gz 2023-06-28T20:09:21.480 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.137039.log.gz 2023-06-28T20:09:21.480 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137373.log 2023-06-28T20:09:21.481 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146162.log 2023-06-28T20:09:21.481 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137373.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-06-28T20:09:21.482 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146162.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.137373.log.gz 2023-06-28T20:09:21.482 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mds.c.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.146162.log.gz 2023-06-28T20:09:21.482 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145498.log 2023-06-28T20:09:21.483 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143587.log 2023-06-28T20:09:21.483 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145498.log: /var/log/ceph/ceph-client.admin.143587.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.145498.log.gz 2023-06-28T20:09:21.483 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.143587.log.gz 2023-06-28T20:09:21.484 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148888.log 2023-06-28T20:09:21.484 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145406.log 2023-06-28T20:09:21.485 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.148888.log: /var/log/ceph/ceph-client.admin.145406.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.148888.log.gz 2023-06-28T20:09:21.485 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.145406.log.gz 2023-06-28T20:09:21.485 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145703.log 2023-06-28T20:09:21.486 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147580.log 2023-06-28T20:09:21.486 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145703.log: /var/log/ceph/ceph-client.admin.147580.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.145703.log.gz 2023-06-28T20:09:21.487 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.147580.log.gz 2023-06-28T20:09:21.487 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144277.log 2023-06-28T20:09:21.487 INFO:teuthology.orchestra.run.smithi043.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-06-28T20:09:21.488 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137419.log 2023-06-28T20:09:21.488 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.144277.log: /var/log/ceph/ceph-client.admin.137419.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.144277.log.gz 2023-06-28T20:09:21.488 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.137419.log.gz 2023-06-28T20:09:21.489 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143541.log 2023-06-28T20:09:21.489 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137513.log 2023-06-28T20:09:21.489 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.143541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137085.log 2023-06-28T20:09:21.490 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.143541.log.gz 2023-06-28T20:09:21.490 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143725.log 2023-06-28T20:09:21.490 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.137513.log.gz 2023-06-28T20:09:21.491 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141633.log 2023-06-28T20:09:21.491 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.143725.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.137085.log.gz 2023-06-28T20:09:21.491 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.143725.log.gz 2023-06-28T20:09:21.492 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141633.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.141633.log.gz 2023-06-28T20:09:21.492 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139374.log 2023-06-28T20:09:21.493 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138060.log 2023-06-28T20:09:21.493 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139374.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.139374.log.gz 2023-06-28T20:09:21.493 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138060.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.138060.log.gz 2023-06-28T20:09:21.494 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138431.log 2023-06-28T20:09:21.494 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146668.log 2023-06-28T20:09:21.494 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138431.log: /var/log/ceph/ceph-client.admin.146668.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.138431.log.gz 2023-06-28T20:09:21.495 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.146668.log.gz 2023-06-28T20:09:21.495 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145024.log 2023-06-28T20:09:21.496 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137810.log 2023-06-28T20:09:21.496 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146714.log 2023-06-28T20:09:21.497 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.145024.log.gz 2023-06-28T20:09:21.497 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137810.log: /var/log/ceph/ceph-client.admin.146714.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.137810.log.gz 2023-06-28T20:09:21.497 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146714.log.gz 2023-06-28T20:09:21.498 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144610.log 2023-06-28T20:09:21.498 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144702.log 2023-06-28T20:09:21.498 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.144610.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.144610.log.gz 2023-06-28T20:09:21.499 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.144702.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.144702.log.gz 2023-06-28T20:09:21.499 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146350.log 2023-06-28T20:09:21.499 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138421.log 2023-06-28T20:09:21.500 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146350.log: /var/log/ceph/ceph-client.admin.138421.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.146350.log.gz 2023-06-28T20:09:21.500 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.138421.log.gz 2023-06-28T20:09:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143407.log 2023-06-28T20:09:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142618.log 2023-06-28T20:09:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.143407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142756.log 2023-06-28T20:09:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142618.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.143407.log.gz 2023-06-28T20:09:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142756.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.142618.log.gz 2023-06-28T20:09:21.502 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.142756.log.gz 2023-06-28T20:09:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145819.log 2023-06-28T20:09:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142332.log 2023-06-28T20:09:21.504 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147239.log 2023-06-28T20:09:21.505 INFO:teuthology.orchestra.run.smithi178.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-06-28T20:09:21.521 INFO:teuthology.orchestra.run.smithi178.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-06-28T20:09:21.529 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142332.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.145819.log.gz 2023-06-28T20:09:21.530 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.142332.log.gz 2023-06-28T20:09:21.530 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147239.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.147239.log.gz 2023-06-28T20:09:21.530 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146580.log 2023-06-28T20:09:21.531 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136626.log 2023-06-28T20:09:21.531 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146580.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.146580.log.gz 2023-06-28T20:09:21.531 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.136626.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.136626.log.gz 2023-06-28T20:09:21.531 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-06-28T20:09:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.141085.log 2023-06-28T20:09:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.2.141085.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.141085.log.gz 2023-06-28T20:09:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145358.log 2023-06-28T20:09:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138424.log 2023-06-28T20:09:21.533 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-06-28T20:09:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142526.log 2023-06-28T20:09:21.533 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4%/var/log/ceph/ceph-client.admin.138424.log: -- replaced with /var/log/ceph/ceph-client.admin.145358.log.gz 2023-06-28T20:09:21.533 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.138424.log.gz 2023-06-28T20:09:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142526.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.142526.log.gz 2023-06-28T20:09:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146852.log 2023-06-28T20:09:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144794.log 2023-06-28T20:09:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148453.log 2023-06-28T20:09:21.535 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.146852.log.gz 2023-06-28T20:09:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.144794.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.144794.log.gz 2023-06-28T20:09:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139604.log 2023-06-28T20:09:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.148453.log: /var/log/ceph/ceph-client.admin.139604.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.148453.log.gz 2023-06-28T20:09:21.536 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141449.log 2023-06-28T20:09:21.536 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.139604.log.gz 2023-06-28T20:09:21.536 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139006.log 2023-06-28T20:09:21.536 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141403.log 2023-06-28T20:09:21.536 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139006.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.141449.log.gz 2023-06-28T20:09:21.537 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.139006.log.gz 2023-06-28T20:09:21.537 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142102.log 2023-06-28T20:09:21.537 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.141403.log.gz--verbose 2023-06-28T20:09:21.537 INFO:teuthology.orchestra.run.smithi043.stderr:-- /var/log/ceph/ceph-client.admin.142844.log 2023-06-28T20:09:21.537 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142102.log: /var/log/ceph/ceph-client.admin.142844.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.142102.log.gz 2023-06-28T20:09:21.538 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.142844.log.gz 2023-06-28T20:09:21.538 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139190.log 2023-06-28T20:09:21.538 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138792.log 2023-06-28T20:09:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141864.log 2023-06-28T20:09:21.539 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.139190.log.gz 2023-06-28T20:09:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138792.log: /var/log/ceph/ceph-client.admin.141864.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.138792.log.gz 2023-06-28T20:09:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143177.log 2023-06-28T20:09:21.540 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.141864.log.gz 2023-06-28T20:09:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145212.log 2023-06-28T20:09:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.143177.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.143177.log.gz 2023-06-28T20:09:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138428.log 2023-06-28T20:09:21.541 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.145212.log.gz-5 2023-06-28T20:09:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:--verbose -- /var/log/ceph/ceph.tmp-client.admin.114756.log 2023-06-28T20:09:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143817.log 2023-06-28T20:09:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph.tmp-client.admin.114756.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.138428.log.gz 2023-06-28T20:09:21.542 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.114756.log.gz 2023-06-28T20:09:21.542 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.143817.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.143817.log.gz 2023-06-28T20:09:21.542 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146442.log 2023-06-28T20:09:21.542 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.146056.log 2023-06-28T20:09:21.542 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147718.log 2023-06-28T20:09:21.543 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.146056.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.146442.log.gz 2023-06-28T20:09:21.543 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.146056.log.gz 2023-06-28T20:09:21.543 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147718.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.147718.log.gz 2023-06-28T20:09:21.543 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146898.log 2023-06-28T20:09:21.543 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142572.log 2023-06-28T20:09:21.544 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146898.log: /var/log/ceph/ceph-client.admin.142572.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.146898.log.gz 2023-06-28T20:09:21.544 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.142572.log.gz 2023-06-28T20:09:21.544 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141587.log 2023-06-28T20:09:21.544 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143909.log 2023-06-28T20:09:21.544 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137467.log 2023-06-28T20:09:21.545 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.143909.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.141587.log.gz 2023-06-28T20:09:21.545 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137467.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.143909.log.gz 2023-06-28T20:09:21.545 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.137467.log.gz 2023-06-28T20:09:21.545 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138791.log 2023-06-28T20:09:21.546 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148315.log 2023-06-28T20:09:21.546 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138791.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.138791.log.gz 2023-06-28T20:09:21.546 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.148315.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.148315.log.gz 2023-06-28T20:09:21.546 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141679.log 2023-06-28T20:09:21.546 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138053.log 2023-06-28T20:09:21.547 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141679.log: /var/log/ceph/ceph-client.admin.138053.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.141679.log.gz 2023-06-28T20:09:21.547 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.138053.log.gz 2023-06-28T20:09:21.547 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144932.log 2023-06-28T20:09:21.547 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138430.log 2023-06-28T20:09:21.547 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.144932.log: /var/log/ceph/ceph-client.admin.138430.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.144932.log.gz 2023-06-28T20:09:21.548 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.138430.log.gz 2023-06-28T20:09:21.548 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-06-28T20:09:21.548 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144388.log 2023-06-28T20:09:21.548 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148499.log 2023-06-28T20:09:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.144388.log: /var/log/ceph/ceph-client.admin.148499.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.144388.log.gz 2023-06-28T20:09:21.549 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.148499.log.gz 2023-06-28T20:09:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139052.log 2023-06-28T20:09:21.549 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137764.log 2023-06-28T20:09:21.550 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139052.log: /var/log/ceph/ceph-client.admin.137764.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.139052.log.gz 2023-06-28T20:09:21.550 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.137764.log.gz 2023-06-28T20:09:21.550 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139558.log 2023-06-28T20:09:21.550 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144748.log 2023-06-28T20:09:21.550 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139558.log: /var/log/ceph/ceph-client.admin.144748.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.139558.log.gz 2023-06-28T20:09:21.551 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.144748.log.gz 2023-06-28T20:09:21.551 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148031.log 2023-06-28T20:09:21.551 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143679.log 2023-06-28T20:09:21.552 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.148031.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.148031.log.gz/var/log/ceph/ceph-client.admin.143679.log: 2023-06-28T20:09:21.552 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.143679.log.gz 2023-06-28T20:09:21.552 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139942.log 2023-06-28T20:09:21.552 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143955.log 2023-06-28T20:09:21.552 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139942.log: 88.9%/var/log/ceph/ceph-client.admin.143955.log: -- replaced with /var/log/ceph/ceph-client.admin.139942.log.gz 2023-06-28T20:09:21.553 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.143955.log.gz 2023-06-28T20:09:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131057.log 2023-06-28T20:09:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138051.log 2023-06-28T20:09:21.553 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.131057.log: /var/log/ceph/ceph-client.admin.138051.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.131057.log.gz 2023-06-28T20:09:21.553 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.138051.log.gz 2023-06-28T20:09:21.554 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141311.log 2023-06-28T20:09:21.554 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142664.log 2023-06-28T20:09:21.554 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141311.log: /var/log/ceph/ceph-client.admin.142664.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.141311.log.gz 2023-06-28T20:09:21.554 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.142664.log.gz 2023-06-28T20:09:21.554 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148131.log 2023-06-28T20:09:21.555 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138056.log 2023-06-28T20:09:21.555 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.148131.log: /var/log/ceph/ceph-client.admin.138056.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.148131.log.gz 2023-06-28T20:09:21.555 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.138056.log.gz 2023-06-28T20:09:21.555 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144434.log 2023-06-28T20:09:21.556 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144139.log 2023-06-28T20:09:21.556 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.144434.log: /var/log/ceph/ceph-client.admin.144139.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.144434.log.gz 2023-06-28T20:09:21.556 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.144139.log.gz 2023-06-28T20:09:21.556 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142002.log 2023-06-28T20:09:21.556 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148935.log 2023-06-28T20:09:21.557 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142002.log: /var/log/ceph/ceph-client.admin.148935.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.142002.log.gz 2023-06-28T20:09:21.557 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.148935.log.gz 2023-06-28T20:09:21.557 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138799.log 2023-06-28T20:09:21.557 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145312.log 2023-06-28T20:09:21.557 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138799.log: /var/log/ceph/ceph-client.admin.145312.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.138799.log.gz 2023-06-28T20:09:21.558 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.145312.log.gz 2023-06-28T20:09:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138058.log 2023-06-28T20:09:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137672.log 2023-06-28T20:09:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138058.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.138058.log.gz 2023-06-28T20:09:21.558 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137672.log: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-06-28T20:09:21.559 INFO:teuthology.orchestra.run.smithi043.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.137672.log.gz 2023-06-28T20:09:21.559 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136937.log 2023-06-28T20:09:21.559 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143315.log 2023-06-28T20:09:21.559 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.136937.log: /var/log/ceph/ceph-client.admin.143315.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.136937.log.gz 2023-06-28T20:09:21.560 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.143315.log.gz 2023-06-28T20:09:21.560 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144001.log 2023-06-28T20:09:21.560 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143495.log 2023-06-28T20:09:21.560 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.144001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148077.log 2023-06-28T20:09:21.561 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.144001.log.gz 2023-06-28T20:09:21.561 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.143495.log: /var/log/ceph/ceph-client.admin.148077.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.143495.log.gz 2023-06-28T20:09:21.561 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148177.log 2023-06-28T20:09:21.561 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.148077.log.gz 2023-06-28T20:09:21.561 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139650.log 2023-06-28T20:09:21.562 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.148177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147533.log 2023-06-28T20:09:21.562 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139650.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.148177.log.gz 2023-06-28T20:09:21.563 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.139650.log.gz 2023-06-28T20:09:21.563 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146396.log 2023-06-28T20:09:21.563 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147533.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.147533.log.gz 2023-06-28T20:09:21.564 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146396.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.146396.log.gz 2023-06-28T20:09:21.564 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145164.log 2023-06-28T20:09:21.564 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143633.log 2023-06-28T20:09:21.565 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.145164.log: 95.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-06-28T20:09:21.565 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.145164.log.gz 2023-06-28T20:09:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.143633.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.143633.log.gz 2023-06-28T20:09:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147469.log 2023-06-28T20:09:21.566 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142890.log 2023-06-28T20:09:21.567 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.146052.log 2023-06-28T20:09:21.567 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.147469.log.gz 2023-06-28T20:09:21.568 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146534.log 2023-06-28T20:09:21.568 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147147.log 2023-06-28T20:09:21.568 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.146052.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.142890.log.gz 2023-06-28T20:09:21.569 INFO:teuthology.orchestra.run.smithi043.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.146052.log.gz 2023-06-28T20:09:21.569 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143771.log 2023-06-28T20:09:21.570 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.146534.log.gz 2023-06-28T20:09:21.570 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139420.log 2023-06-28T20:09:21.570 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.147147.log.gz 2023-06-28T20:09:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.143771.log: /var/log/ceph/ceph-client.admin.139420.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.143771.log.gz 2023-06-28T20:09:21.571 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.139420.log.gz 2023-06-28T20:09:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139144.log 2023-06-28T20:09:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138054.log 2023-06-28T20:09:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148361.log 2023-06-28T20:09:21.573 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.139144.log.gz 2023-06-28T20:09:21.573 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.138054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137319.log 2023-06-28T20:09:21.573 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136884.log 2023-06-28T20:09:21.574 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.148361.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.138054.log.gz 2023-06-28T20:09:21.574 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138793.log 2023-06-28T20:09:21.575 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.148361.log.gz 2023-06-28T20:09:21.575 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.136884.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137319.log.gz 2023-06-28T20:09:21.575 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144656.log 2023-06-28T20:09:21.576 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7%/var/log/ceph/ceph-client.admin.138793.log: -- replaced with /var/log/ceph/ceph-client.admin.136884.log.gz 2023-06-28T20:09:21.576 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.144656.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.138793.log.gz 2023-06-28T20:09:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144047.log 2023-06-28T20:09:21.577 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.144656.log.gz 2023-06-28T20:09:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147423.log 2023-06-28T20:09:21.578 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.144047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145452.log 2023-06-28T20:09:21.578 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.144047.log.gz 2023-06-28T20:09:21.578 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147423.log: /var/log/ceph/ceph-client.admin.145452.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.147423.log.gz 2023-06-28T20:09:21.578 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.145452.log.gz 2023-06-28T20:09:21.579 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137271.log 2023-06-28T20:09:21.579 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-06-28T20:09:21.579 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131498.log 2023-06-28T20:09:21.579 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141910.log 2023-06-28T20:09:21.579 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.137271.log.gz 2023-06-28T20:09:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.131498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146963.log 2023-06-28T20:09:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141910.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.131498.log.gz 2023-06-28T20:09:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146963.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.141910.log.gz 2023-06-28T20:09:21.581 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.146963.log.gz 2023-06-28T20:09:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147055.log 2023-06-28T20:09:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141772.log 2023-06-28T20:09:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148223.log 2023-06-28T20:09:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.141772.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.147055.log.gz 2023-06-28T20:09:21.582 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.141772.log.gz 2023-06-28T20:09:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.148223.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.148223.log.gz 2023-06-28T20:09:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144840.log 2023-06-28T20:09:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144978.log 2023-06-28T20:09:21.582 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.144840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139236.log 2023-06-28T20:09:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.144978.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.144840.log.gz 2023-06-28T20:09:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.139236.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.144978.log.gz 2023-06-28T20:09:21.583 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.139236.log.gz 2023-06-28T20:09:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146488.log 2023-06-28T20:09:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147985.log 2023-06-28T20:09:21.583 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.146488.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-06-28T20:09:21.584 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147985.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.146488.log.gz 2023-06-28T20:09:21.584 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mgr.y.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.147985.log.gz 2023-06-28T20:09:21.584 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142710.log 2023-06-28T20:09:21.584 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148269.log 2023-06-28T20:09:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.142710.log: /var/log/ceph/ceph-client.admin.148269.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.142710.log.gz 2023-06-28T20:09:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137177.log 2023-06-28T20:09:21.585 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.148269.log.gz 2023-06-28T20:09:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.137177.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.137177.log.gz 2023-06-28T20:09:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147377.log 2023-06-28T20:09:21.585 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.147377.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.147377.log.gz 2023-06-28T20:09:21.586 INFO:teuthology.orchestra.run.smithi043.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-06-28T20:09:21.587 INFO:teuthology.orchestra.run.smithi043.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-06-28T20:09:21.587 INFO:teuthology.orchestra.run.smithi043.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-06-28T20:09:21.588 INFO:teuthology.orchestra.run.smithi178.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-06-28T20:09:21.588 INFO:teuthology.orchestra.run.smithi178.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-06-28T20:09:21.588 INFO:teuthology.orchestra.run.smithi178.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-06-28T20:09:21.588 INFO:teuthology.orchestra.run.smithi178.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-06-28T20:09:21.589 INFO:teuthology.orchestra.run.smithi178.stderr: 2023-06-28T20:09:21.589 INFO:teuthology.orchestra.run.smithi178.stderr:real 0m0.147s 2023-06-28T20:09:21.589 INFO:teuthology.orchestra.run.smithi178.stderr:user 0m0.659s 2023-06-28T20:09:21.589 INFO:teuthology.orchestra.run.smithi178.stderr:sys 0m0.039s 2023-06-28T20:09:21.590 INFO:teuthology.orchestra.run.smithi043.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-06-28T20:09:21.691 INFO:teuthology.orchestra.run.smithi043.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-06-28T20:09:21.693 INFO:teuthology.orchestra.run.smithi043.stderr: 2023-06-28T20:09:21.694 INFO:teuthology.orchestra.run.smithi043.stderr:real 0m0.314s 2023-06-28T20:09:21.694 INFO:teuthology.orchestra.run.smithi043.stderr:user 0m0.873s 2023-06-28T20:09:21.694 INFO:teuthology.orchestra.run.smithi043.stderr:sys 0m0.175s 2023-06-28T20:09:21.695 INFO:tasks.ceph:Archiving logs... 2023-06-28T20:09:21.696 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-06-28_10:04:29-fs-wip-vshankar-testing-20230628.043830-testing-default-smithi/7319991/remote/smithi043/log 2023-06-28T20:09:21.701 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/log/ceph -- . 2023-06-28T20:09:23.616 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-06-28_10:04:29-fs-wip-vshankar-testing-20230628.043830-testing-default-smithi/7319991/remote/smithi178/log 2023-06-28T20:09:23.618 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/log/ceph -- . 2023-06-28T20:09:23.732 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-06-28T20:09:23.753 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-06-28T20:09:23.754 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-06-28T20:09:23.756 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-06-28T20:09:31.523 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-06-28T20:09:31.534 INFO:teuthology.task.clock:Checking final clock skew... 2023-06-28T20:09:31.535 DEBUG:teuthology.orchestra.run.smithi043:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-28T20:09:31.541 DEBUG:teuthology.orchestra.run.smithi178:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-28T20:09:31.555 INFO:teuthology.orchestra.run.smithi043.stderr:bash: ntpq: command not found 2023-06-28T20:09:31.559 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-28T20:09:31.560 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2023-06-28T20:09:31.560 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 9 +397us[ +397us] +/- 88ms 2023-06-28T20:09:31.561 INFO:teuthology.orchestra.run.smithi043.stdout:^* hv02.front.sepia.ceph.com 3 6 377 11 -1044us[-1043us] +/- 53ms 2023-06-28T20:09:31.561 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 +1176us[+1176us] +/- 77ms 2023-06-28T20:09:31.561 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-06-28T20:09:31.562 INFO:teuthology.orchestra.run.smithi178.stderr:bash: ntpq: command not found 2023-06-28T20:09:31.565 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-28T20:09:31.566 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2023-06-28T20:09:31.566 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 9 +418us[ +418us] +/- 88ms 2023-06-28T20:09:31.566 INFO:teuthology.orchestra.run.smithi178.stdout:^* hv02.front.sepia.ceph.com 3 6 377 11 -1038us[-1044us] +/- 53ms 2023-06-28T20:09:31.566 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 +1197us[+1191us] +/- 77ms 2023-06-28T20:09:31.567 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-06-28T20:09:31.568 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-06-28T20:09:31.658 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-06-28T20:09:31.658 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-06-28T20:09:31.670 DEBUG:teuthology.orchestra.run.smithi043:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-06-28T20:09:31.779 DEBUG:teuthology.orchestra.run.smithi178:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-06-28T20:09:31.879 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-28T20:09:31.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-28T20:09:31.919 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-28T20:09:31.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-28T20:09:31.952 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-06-28T20:09:31.963 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-06-28T20:09:31.972 INFO:teuthology.task.internal:Duration was 1275.416331 seconds 2023-06-28T20:09:31.973 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-06-28T20:09:31.991 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-06-28T20:09:31.991 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-06-28T20:09:31.994 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-06-28T20:09:32.041 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-28T20:09:32.042 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-28T20:09:32.391 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-06-28T20:09:32.391 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi043.front.sepia.ceph.com 2023-06-28T20:09:32.392 DEBUG:teuthology.orchestra.run.smithi043:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-06-28T20:09:32.415 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi178.front.sepia.ceph.com 2023-06-28T20:09:32.415 DEBUG:teuthology.orchestra.run.smithi178:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-06-28T20:09:32.438 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-06-28T20:09:32.439 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-28T20:09:32.457 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-28T20:09:32.518 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-06-28T20:09:32.519 DEBUG:teuthology.orchestra.run.smithi043:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-28T20:09:32.541 DEBUG:teuthology.orchestra.run.smithi178:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-28T20:09:32.633 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-06-28T20:09:32.645 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-06-28T20:09:32.645 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-06-28T20:09:32.673 DEBUG:teuthology.orchestra.run.smithi178:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-06-28T20:09:32.700 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-06-28T20:09:32.711 DEBUG:teuthology.orchestra.run.smithi043:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-06-28T20:09:32.715 DEBUG:teuthology.orchestra.run.smithi178:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-06-28T20:09:32.739 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = core 2023-06-28T20:09:32.767 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = core 2023-06-28T20:09:32.780 DEBUG:teuthology.orchestra.run.smithi043:> test -e /home/ubuntu/cephtest/archive/coredump 2023-06-28T20:09:32.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-28T20:09:32.812 DEBUG:teuthology.orchestra.run.smithi178:> test -e /home/ubuntu/cephtest/archive/coredump 2023-06-28T20:09:32.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-28T20:09:32.837 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-06-28T20:09:32.848 INFO:teuthology.task.internal:Transferring archived files... 2023-06-28T20:09:32.849 DEBUG:teuthology.misc:Transferring archived files from smithi043:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-06-28_10:04:29-fs-wip-vshankar-testing-20230628.043830-testing-default-smithi/7319991/remote/smithi043 2023-06-28T20:09:32.849 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-06-28T20:09:32.890 DEBUG:teuthology.misc:Transferring archived files from smithi178:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-06-28_10:04:29-fs-wip-vshankar-testing-20230628.043830-testing-default-smithi/7319991/remote/smithi178 2023-06-28T20:09:32.890 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-06-28T20:09:32.930 INFO:teuthology.task.internal:Removing archive directory... 2023-06-28T20:09:32.931 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf -- /home/ubuntu/cephtest/archive 2023-06-28T20:09:32.933 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf -- /home/ubuntu/cephtest/archive 2023-06-28T20:09:32.979 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-06-28T20:09:32.991 INFO:teuthology.task.internal:Not uploading archives. 2023-06-28T20:09:32.992 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-06-28T20:09:33.007 INFO:teuthology.task.internal:Tidying up after the test... 2023-06-28T20:09:33.007 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-06-28T20:09:33.010 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-06-28T20:09:33.024 INFO:teuthology.orchestra.run.smithi043.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 28 20:09 /home/ubuntu/cephtest 2023-06-28T20:09:33.041 INFO:teuthology.orchestra.run.smithi178.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 28 20:09 /home/ubuntu/cephtest 2023-06-28T20:09:33.041 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-06-28T20:09:33.051 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-06-28T20:09:33.079 INFO:teuthology.nuke:Checking targets against current locks 2023-06-28T20:09:38.110 DEBUG:teuthology.nuke:shortname: smithi043 2023-06-28T20:09:38.111 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-28T20:09:48.145 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-06-28_10:04:29-fs-wip-vshankar-testing-20230628.043830-testing-default-smithi/7319991', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-28 19:41:37.828967', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-06-28T20:09:53.154 DEBUG:teuthology.nuke:shortname: smithi178 2023-06-28T20:09:53.155 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-28T20:09:58.193 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-06-28_10:04:29-fs-wip-vshankar-testing-20230628.043830-testing-default-smithi/7319991', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-28 19:41:37.830701', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-06-28T20:09:58.233 INFO:teuthology.orchestra.console:Power off smithi043 2023-06-28T20:09:58.235 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-28T20:09:58.262 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T20:09:58.285 INFO:teuthology.orchestra.console:Power off smithi178 2023-06-28T20:09:58.286 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-28T20:09:58.317 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T20:10:02.304 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T20:10:02.343 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T20:10:06.424 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T20:10:06.459 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T20:10:11.549 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T20:10:11.573 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T20:10:16.668 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T20:10:16.687 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T20:10:21.788 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T20:10:21.809 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T20:10:26.912 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T20:10:26.924 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi178.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T20:10:27.143 INFO:teuthology.orchestra.console:Power off for smithi178 completed 2023-06-28T20:10:32.027 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-28T20:10:32.241 INFO:teuthology.orchestra.console:Power off for smithi043 completed 2023-06-28T20:10:32.356 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} duration: 1275.4163308143616 failure_reason: 'Test failure: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)' flavor: default owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=f4f09624390e49ceb6a0d3bf9219ac27 status: fail success: false 2023-06-28T20:10:32.357 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-28T20:10:36.905 INFO:teuthology.run:FAIL